Support python -Xcpu_count=<n> feature for container environment.

[corona10]

Feature or enhancement

As #80235, there are requests for isolating CPU count in k8s or container environment, and this is a very important feature these days. (Practically my corp, a lot of workloads are running under container environments, and controlling CPU count is very important to resolve busy neigrhood issues)

There were a lot of discussions, and following the cgroup spec requires a lot of complexity and performance issues (due to fallback).
JDK 21 chooses not to depend on CPU Shares to compute active processor count and they choose to use -XX:ActiveProcessorCount=<n>.
see: https://bugs.openjdk.org/browse/JDK-8281571

I think that this strategy will be worth using from the CPython side too.
So if the user executes the python with -Xcpu_count=3option os.cpu_count will return 3, instead of the actual CPU count that was calculated from os.cpu_count.

cc @vstinner @indygreg

Linked PRs

• gh-109595: Add -Xcpu_count=<n> cmdline for container users #109667
• gh-110649: Update -Xcpu_count=<n> cmdline to support process mode #110639

[vstinner]

I would prefer to first fix os.cpu_count() to take sched_getaffinity() in account. But we should provide the two values: taking affinity in account, and "total" number of CPUs. It might be tricky to change the default :-(

[corona10]

It might be tricky to change the default :-(

Providing -Xcpu_count will not touch any default behavior until the user explicitly set the value. So it will be the expected side-effect from the user side.

I would prefer to first fix os.cpu_count() to take sched_getaffinity() in account

We need to understand the actual user's usecase. AFAIK k8s users or container users never use something like taskset to isolate CPU resources. They just write a Docker file and set the resources to k8s pods.
Even if we update os.cpu_count to reflect sched_getaffinity, it will not be directly helpful to actual container people.

I think that that's why JDK still provides -XX:ActiveProcessorCount=<n>.
Until we do not provide a practical way to limit CPU resources, there is no way to reflect os.cpu_count to follow their limitation.
And as you know, container application is veeeery important use case in modern applications.

[vstinner]

os.cpu_count() is used in other use cases than containers, where CPU affinity is used.

If we modify cpu_count() to take affinity in account and/or if your -X option is implemented, we should add an option to get "total number of CPUs".

[indygreg]

AFAIK k8s users or container users never use something like taskset to isolate CPU resources.

I have used taskset in Kubernetes. It is useful to pin processes to specific CPU cores without using something more advanced/modern like the CPU Manager feature.

One of the things I used it for was testing the behavior of a JVM when passing -XX:ActiveProcessorCount= to a different value than the number of CPUs being allowed to schedule on.

There's also the scenario where you have a container running multiple processes and don't want a process to use all available CPUs. In this scenario, in the absence of [something like] taskset you need something like -XX:ActiveProcessorCount= to forcefully override the default heuristics. (This use case is important for people who realize how inefficient the artificial containers are only 1 process rule is and have chosen to eschew it.)

[vstinner]

I created issue #109649: os.cpu_count(): add "affinity" parameter to get the number of "usable" CPUs.

UPDATE: I renamed the usable parameter to affinity.

[corona10]

There's also the scenario where you have a container running multiple processes and don't want a process to use all available CPUs. In this scenario, in the absence of [something like] taskset you need something like -XX:ActiveProcessorCount= to forcefully override the default heuristics. (This use case is important for people who realize how inefficient the artificial containers are only 1 process rule is and have chosen to eschew it.)

Yeah, this is exactly what I want to say :) Thank you for emphasizing!

[corona10]

The reason I don't support cgroup directly is the following reasons.

• cgroups 've existed even before docker or podman borned: https://docs.kernel.org/admin-guide/cgroup-v1/cgroups.html
• so there are few variation of representing CPU resources and it makes hard to decide which value that the user want to use

docker run ... --cpu-shares=512 java .... ==> os::active_processor_count() = 1
docker run ... --cpu-shares=1024 java ... ==> os::active_processor_count() = 32 (total CPUs on this system)
docker run ... --cpu-shares=2048 java ... ==> os::active_processor_count() = 2

• It will increase overhead itself to os.cpu_count itself, if we decide to parse cgroup, we need to parse cgroup anyway by default, AFAIK there is no way to avoid this path if we decide to support cgroup as seamless approach to what people really want.
• Supporting cgroupv2 is much harder than cgroupv1, it will increase the maintenance cost for cpython codebase, and we need to follow up new specifications if cgroupv3 is released.
• Let's not follow what JDK already tried and failed: https://bugs.openjdk.org/browse/JDK-8281571
• Adding a new flag to os.cpu_count() will not propagate to all libraries that the program depends on but this solution will do.

[indygreg]

Is there an appetite for allowing non-integer / special values to -Xcpu_count= that define the mechanism to derive the CPU count? e.g.

• -Xcpu_count=system-processor-count would use the equivalent of os.cpu_count() today.
• -Xcpu_count=cgroups-shares-1024 would read from cgroups cpu.shares and divide by 1024.
• -Xcpu_count=sched-affinity would compute CPU count from scheduler affinity.

Obviously we would need to bikeshed the names and behavior bit. But I see this as a potential elegant solution that not only allows forceful count overrides but also allows dynamic derivation using well-defined semantics. It also gives CPython more flexibility to introduce new variants/behavior without breaking backwards compatibility. It seemingly placates all parties.

[vstinner]

-Xcpu_count=sched-affinity would compute CPU count from scheduler affinity.

As the author of PR #109649, I'm interested by this mode 😁