sc_cor_qt::stack_protect

[tdoherty]

Hello all,

 

I've recently created a vary large model and I have encounted an assertion violation in sc_cor_qt::stack_protect.  The mprotect system call returns a -1 value, triggering the assertion. The errno is ENOMEM. There is no memory allcoated expilicitly in stack_protect so it must mean that the address passed to mprotect is invalid.

 

If I remove the stack_protect call from sc_thread_process::prepare_for_simulation then my model runs just fine. So it appears that stack_protect is not strictly needed, but probably a good idea to leave it in if you can.

 

Anyone else ever have any problems with the stack_protect call?

 

Thanks,

Terry

[Philipp A Hartmann]

First of all, it is recommended to keep the stack protection activated (where available) to avoid arbitrary memory corruption upon stack overflow.  Stack-related memory corruption is very hard to debug...

 

Assuming, you're running on Linux, then secondly, there are indeed (kernel) resources allocated within the mprotect call.  Quoting the mprotect(2) manpage:

ERRORS
...
       ENOMEM Internal kernel structures could not be allocated.

       ENOMEM Addresses in the range [addr, addr+len-1] are invalid for the address space of the  process,  or
              specify  one  or  more  pages  that are not mapped.  (Before kernel 2.4.19, the error EFAULT was
              incorrectly produced for these cases.)

Activating the memory protection on some page(s) requires to setup an appropriate memory mapping, of which a limited number can be created by the kernel.  You can try to increase this limit by

$ sudo bash -c 'echo 131060 > /proc/sys/vm/max_map_count'

(which doubles the default number on my system).

 

Last but not least, some additional questions:

• Do you encounter the problem during elaboration or during simulation?
  If it occurs during simulation, you may have "leaking" dynamic processes. If possible, try to reuse dynamic process instances you've created.

• Which version of SystemC are you using?
  There has been an internal "dynamic process/object leak" in some cases, which has been fixed in SystemC 2.3.0.

• Can you change some thread processes to method processes?
  SC_METHODs have a significantly lower simulation overhead.  Especially, when you're using thousands of processes, this may be a worthy optimization.

Hope that helps,
  Philipp

[tdoherty]

Thank you for the reply Philipp.

 

I am currenlty using SystemC 2.3.0 on a 64 bit linux system with great quantities of physical memory.

 

I believe the assertion occurs before simulation, because stack_protect is called from sc_thread_process::prepare_for_simulation. 

 

A lack of memory resources during the mprotect call seems curious. If I run my simulation with around 6000 instances of this sc_module I do not get the assertion violation. If  I increase the number of instances to 8000 or so then I do get the assertion violation.  When I comment out the call to stack_protect I am able to run with 8000 instances seemingly without error implying something could be awry, but it is not manifesting itself.

 

I did reduce the number of threads per instance from 4 to 2 to attempt to solve the issue, but it did not help. Instantiating over 6000 still resulted in an assertion violation regardless of 2 or 4 threads.  If it is hitting the max_map_count value I would think that reducing the number of threads would help.  I need to do some more investigating.

 

Thank you again,

Terry

[Philipp A Hartmann]

I believe the assertion occurs before simulation, because stack_protect is called from sc_thread_process::prepare_for_simulation.

 

Of course, sc_thread_process::prepare_for_simulation is called for dynamic processes as well, see sc_simcontext::create_[c]thread_process. But you should be able to observe the start of the simulation before the assertion fails, in case of a later resource exhaustion being the problem.

 

I did reduce the number of threads per instance from 4 to 2 to attempt to solve the issue, but it did not help. Instantiating over 6000 still resulted in an assertion violation regardless of 2 or 4 threads.  If it is hitting the max_map_count value I would think that reducing the number of threads would help.  I need to do some more investigating.

 

You don't provide sufficient information about what kind of module you instantiate 6000-8000 times.  Based on your comments, you seem to have at least 12000 threads in your simulation, which indeed seems to be quite many.  You could try to add a simple (static) counter to the stack_protect call and print out a message for the first failing mprotect call to shed some more light on this issue.  My guess would still be that some OS resources are running low.

 

/Philipp

[tdoherty]

Hi Philipp,

 

I am not using dynamic threads in this instance so the issue does occur before simulation.

 

It is probably the a system resource limit.

 

Thank you for the help.

Terry