Nuwa

There's a process in Firefox OS similar to Android's Zygote, which uses fork() to spawn application processes and let them share memory pages by copy-on-write. It helps on lowering memory requirement and faster application launching. It is named Nuwa [1], a goddess in ancient Chinese mythology.

When look at it closer, you will find out things are not as easy as simply calling fork(). For instance, the threads created before fork() in Nuwa process are not inherited in child process; the open file descriptors for epoll from Nuwa process can't be used by child processes for IPC. We need to deal with those issues before we can earn some memory.

I'll try to explain how threads are recreated in child process.

First of all, we use the wrap feature [2] of ld to wrap some pthread functions. For example, all calls to pthread_create() will call the function __wrap_pthread_create() defined in Nuwa.cpp [3] instead.

When pthread_create() is called in Nuwa process, __wrap_pthread_create() new a thread info structure to store related data and keeps it in a linked list sAllThreads. Also a stack is allocated for the pthread being created. The actual start routine is then called in _thread_create_startup().

Once the thread runs and calls any of epoll_wait(), poll(), pthread_mutex_lock(), pthread_cond_wait(), and pthread_cond_timedwait(), it will setjmp() and call pthread_mutex_lock() on an already locked mutex to block the thread. Those are done in the corresponding wrappers by macro THREAD_FREEZE_POINT*.

Later when a child process is forked, RecreateThreads() iterates through the linked list sAllThreads in child process and invokes pthread_create() on each TINFO_FLAG_NUWA_SUPPORT flagged one to recreate them. The startup function here is thread_recreate_startup(), which restores thread name, remember recreated pthread ID and native thread ID, setjmp(), and then longjmp() to the place where the thread setjmp() earlier in Nuwa process to continue the thread. This is also why a malloced stack is used, it is necessary for keeping the stack and resuming the thread in child process. After the thread routine terminates, it returns to thread_create_startup() and then longjmp() back to where it setjmp() in thread_recreate_startup(). Here's an image for better understanding:

You may notice there are some other wrapped pthread functions as well, for example: pthread_join(). It is wrapped because the ID of pthread created in Nuwa process may be stored somewhere and inherited in child process. If child process calls pthread_join() with the ID, it needs to map to the recreated one, otherwise there'll be an error.

I hope this shed some light. :)

[1] http://en.wikipedia.org/wiki/N%C3%BCwa
[2] http://ftp.gnu.org/pub/old-gnu/Manuals/ld-2.9.1/html_node/ld_3.html
[3] http://hg.mozilla.org/mozilla-central/file/7297cfffd91c/mozglue/build/Nuwa.cpp