Yes, these tasks are completely independent once started. And processes are easier to clean up (done by the OS at exit). But the multiprocessing package seems more focused on all the communication and sharing support that I don’t need.

I am thinking, if socket passing works well in this, of having a child do the listen loop and pass everything to the parent, which then makes new child workers as needed. But even this runs into a classic programming issue of how to wait for 2 things at the same time when each provides its own function call to wait for one thing.

You want to spawn more independent threads that do not need much interaction. If you write the code to handle the connections after they come in as a separate program you could look at the subprocess module to manage them. If you want them to be more conventionally part of your main program you may want to look at threading which permits more control over independent threads or event he lower level thread module instead of using multiprocessing.

I want to create a network daemon that listens for new incoming connections. For each connection that arrives, I want to run a specific handler for that connection (a worker process). This will need to be sure the connection resources (and underlying system socket) are closed in the parent process, and everything else is closed in the child process. In the C language this is a cumbersome chore. I’m hoping the multiprocessing package has a means to handle this.

Once the child/worker is running, I have no further need to communicate or share with it. The parent should resume listening for more connections. So calling join() would break the purpose, which is to allow many processes for many connections, without the time taken by one to delay any others. Fortunately for my case these are not computational, so I could run many worker processes per CPU core.

But I would like keep track of how many are running so the parent can stop listening if the number of workers is at the maximum (for example 100 workers might be the maximum). This would mean the multiprocessing package would be handling the child process exits somehow behind the scenes. The parent would just wait until the number of children get below 100 then resume the listen loop.

I can write this in C. But I want to do this in Python so the worker logic can be implemented in Python.

Any ideas about this? BTW, the worker logic is going to do a lot of DB accesses and output some summary results as a very simple HTML page.

Nice article, Timothy. And truer words never spoken.

Also, thank you for the mention. I truly appreciate it.