E549, uncaught exception bad_alloc when invoking sc_spawn()

[arhdr_sc]

 Hello All,

 

I need to spawn a large number of threads for my simulation using sc_spwan(). My threads
are very small piece of code. When I have 16384 threads the simulation stops by issuing
the following message in iteration 5504 of the loop invoking sc_spawn():
 

Error: (E549) uncaught exception: std::bad_alloc
In file: ../../../../src/sysc/kernel/sc_except.cpp:98

I know this happens due to stack size, but I don't know how much memory is used by each
thread.

 

Q1) Is there a way to get the size of stack for each thread? Is there a method/function
like get_stack_size()?

Q2) How can I use set_stack_size() when invoking sc_spawn()?

Q3) My simulation is run in CygWin under Windows 7 (32-bit). Does 32-bit the OS impose a limitation
to the stack size? In other words, can the problem be fixed using a 64-bit OS?

Thank you so much in advance!
 

 

[Philipp A Hartmann]

Q1) Is there a way to get the size of stack for each thread? Is there a method/function

like get_stack_size()?

No.  In SystemC 2.3.1, the platform-specific defaults are defined in sc_thread_process.h.  To obtain the default value used on your platform, you can query the SC_DEFAULT_STACK_SIZE constant in your application.

 

 

Q2) How can I use set_stack_size() when invoking sc_spawn()?

Use sc_spawn_options, see Section 5.5 in IEEE 1666-2011:

sc_spawn_options opt;
  opt.set_stack_size(0x10000);
sc_spawn( func, "func", &opt );

Q3) My simulation is run in CygWin under Windows 7 (32-bit). Does 32-bit the OS impose a limitation

to the stack size? In other words, can the problem be fixed using a 64-bit OS?

 

Yes, on 32-bit OSs, the overall memory space for user applications is usually limited to 2GB (on Windows) and 3GB (on Linux).  On 64-bit OSs, this limitation is lifted.

 

hth,

  Philipp

Edited August 13, 2014 by Philipp A. Hartmann

[arhdr_sc]

Hello,

 

Philipp, thank you very much for your reply.

 

I tried sc_spawn_options you mentioned, however, I got the following error:

 

assertion "m_stack_size > ( 2 * pagesize )" failed: file "../../../../src/sysc/kernel/sc_cor_qt.cpp", line 76, function: virtual void sc_core::sc_cor_qt::stack_protect(bool)

 

When I set the stack size to 0x10000 and 327680 >> 2, I get the above error message (SC_DEFAULT_STACK_SIZE is 327680 for my case). I need much less than this stack size, say, 0x1000 is enough I guess.

 

I looked at the sc_cor_qt::stack_protect() method. I noticed a comment at the top of this method saying:

 

    // Code needs to be tested on HP-UX and disabled if it doesn't work there
    // Code still needs to be ported to WIN32
 

So, How can I set the stack size to 0x1000 without getting error messages.

 

Thanks,

 

Alireza

 

 

 

[Philipp A Hartmann]

Short answer: You can't.

 

The current proof-of-concept simulator employs a stack protection mechanism to detect stack overflows by using the OSs memory protection mechanisms.  This requires an additional memory page (as a red zone) after the "real" stack.  As you have seen, this requires the minimum stack size to exceed a certain value.  0x1000 is certainly too small here. I would have expected that 0x10000 could work, but this seems not to be the case.

 

If your threads are so small and simple, maybe you can port them to SC_METHODs.  Alternatively, you can try to switch to a 64-bit OS.

 

/Philipp

[Philipp A Hartmann]

Oh, I should have mentioned that you can try to build your SystemC kernel with a Pthread threading implementation:

../configure [...] --enable-pthreads

The stack size requirements might be different there.

 

Alternatively, you can use a MinGW32 compiler (http://mingw-w64.sourceforge.net/), which then uses the native Windows threading facilities (WinFiber) under the hood.

 

hth,
Philipp

[arhdr_sc]

 Hi Philipp,

I tried the SystemC installation with Pthread implementation, however, I got the following error message:

"ERROR - could not create thread", which I believe is issued by sc_cor_pkg_pthread::create().

Also, I cannot use SC_METHODs as there is communication between threads using wait().

I am trying to switch to a 64-bit OS. However, it will take some time, as I need to re-install my whole system and application software to the new OS to be able to continue developing my project under the new system.

 

BTW, I noticed a note by you from previous posts (see below) saying "Secondly, you should _NOT_
use pthreads for your SystemC processes unless needed for very specific reasons".
Could you explain why?

http://www.accellera.org/Discussion_Forums/systemc-forum/archive/msg?list_name=systemc-forum&monthdir=201209&msg=msg00028.html

 

Anyways, thank you so much for your helpful and insightful comments.

Regards,

Alireza
 

[Philipp A Hartmann]

I tried the SystemC installation with Pthread implementation, however, I got the following error message:

"ERROR - could not create thread", which I believe is issued by sc_cor_pkg_pthread::create().

 

You could try to print the return value of the pthread_create call via strerror (untested):

    int notok = pthread_create( &cor_p->m_thread, &attr,
             &sc_cor_pthread::invoke_module_method, (void*)cor_p );
    if ( notok )
    {
        std::fprintf(stderr, "ERROR - could not create thread: %s\n", strerror(notok) );
    }

You either set a too small stack size (EINVAL), or are running out of resources (number of allowed threads, memory limits) (EAGAIN).

 

 

Also, I cannot use SC_METHODs as there is communication between threads using wait().

 

You may need to rethink your modeling approach, given the huge number of processes you have.

 

 

I am trying to switch to a 64-bit OS. However, it will take some time, as I need to re-install my whole system and application software to the new OS to be able to continue developing my project under the new system.

 

Before reinstalling your OS, you can try to use a MinGW compiler, as mentioned in my previous answer.  If you run configure from within your Cygwin environment, specify the MinGW target and the MinGW compiler on the command-line:

../configure --target=i686-pc-mingw32 CXX=/path/to/mingw32/g++.exe

BTW, I noticed a note by you from previous posts (see below) saying "Secondly, you should _NOT_

use pthreads for your SystemC processes unless needed for very specific reasons".

Could you explain why?

 

 

Because performance-wise, Cygwin with Pthreads is probably the worst combination on Windows platforms.

 

 

hth,

  Philipp

[arhdr_sc]

Hi Philipp,

==========
Section 1:
==========
About using the MinGW compiler, I noticed that I should change my code as I used
sockets for communication with another program. So, for Windows I need to rewrite my
code using Windows socket API, which I prefer not to do for now.

==========
Section 2:
==========
About rethinking my modeling approach for a large number of threads, I have some
ideas but it needs checkpointing the threads (saving and restoring threads' context
and swap the threads in when they are needed to execute), which I believe the
primitives are not provided by SystemC.

I have a question about this as there is potential to simplify my model to use a small
number of SystemC dynamic threads. I should explain the model first. Suppose I have
N X M serial threads that can be assigned to M parallel threads, each executing N serial
threads. Also, each serial thread is divided into P serial phases. At any point in time,
there are M parallel threads running. Each parallel thread is executing one phase
of one of its serial threads. Once a parallel thread is finished executing its current
serial phase, it switches to the next phase, which can be a phase from another
serial thread assigned to this parallel thread. To clarify I give the following
example with M, N, and P set as follows:

M = 2 (number of parallel threads)
N = 3 (number of serial threads assigned to each parallel thread)
P = 4 (number of serial phases per each serial thread)

Serial threads assigned to parallel threads as follows (this assignment varies
for different simulations):

PT0        PT1
------------------
ST0        ST1
ST2        ST3
ST4        ST5

Parallel thread schedule:

PT0: ST0_PH0, ST2_PH0, ST4_PH0, ST0_PH1, ST2_PH1, ST4_PH1,
         ST0_PH2, ST2_PH2, ST4_PH2, ST0_PH3, ST2_PH3, ST4_PH3;

PT0: ST1_PH0, ST3_PH0, ST5_PH0, ST1_PH1, ST3_PH1, ST5_PH1,
         ST1_PH2, ST3_PH2, ST5_PH2, ST1_PH3, ST3_PH3, ST5_PH3;

Legend:
PT: parallel thread
ST: serial thread
PH: phase
STx_PHy: phase y from serial thread x

For example, PT0 starts with PH0 from ST0, and continues with PH0 from ST2, ...,
until it gets to PH3 from ST4 as a last phase.


==========
Section 3:
==========
About switching to a 64-bit OS, I was able to run my simulation for 16384 threads
on a 64-bit Linux OS. However, when I ran my simulation for (16384 X 4) threads,
I got the following error message:

../../../../src/sysc/kernel/sc_cor_qt.cpp:107: virtual void sc_core::sc_cor_qt::
stack_protect(bool): Assertion `ret == 0' failed.

So, if I could find a solution for reducing the number of active threads as explained
in Section 2, it would be so great.

Sorry for being too long in this post.

Thank you so much.

Regards,

Alireza
 

[Philipp A Hartmann]

First of all, it somewhat depends on the preemption scheme you need to support.  Are the serial threads preemptible while running one of their phases?  If not (and it looks like this in your description), you can refactor the model to only use M "real" SystemC threads, one for each parallel thread.

The serial threads would then have an explicit "state" (i.e. the functional state and the phase they're) and the phases could be implemented as individual functions.  When a serial thread is resumed, you can simply call the corresponding phase function from its assigned "parallel thread".  Details left to the reader. ;-)

 

hth,
  Philipp