Supporting Quickthreads for SC_THREADS on win64

[mo_ayman]

Hi all,

So, I'll start by stating the problem. The problem is that it is not possible to use Quickthreads on 64 bit Windows.

Basically, for sake of consistency with other platforms as well as 32 bit Windows, I need to be able to use Quickthreads as well.

 

So, before I begin I have a question:

• What is the out of the box choice for SystemC threads on Windows platforms both 32 bit and 64 bit?
  • As far as I understand from the configure script, WinFibers is what will be used for Windows if MinGW is being used:

    •      # use WinFiber on MinGW platforms
           AS_CASE([${TARGET_ARCH}],
             [mingw*],[QT_ARCH="none"])

  • Can someone please confirm this?

 

So, I recently had the discussion in this post https://forums.accellera.org/topic/5627-systemc-231-installation-problem-on-windows-7-with-cygwin.

I became sure that the issue is coming from Quickthreads and started trying to get it to work.

Starting from the 64 bit Linux Quickthreads source iX86_64.s, I created a new file x64.s which has the modifications necessary to fit the Windows 64 bit ABI/Calling convention.

I mostly got it to work. I, however, faced an issue related to exception handling.

The issue arises from the throw statements found in sc_thread_process.h which affect most of sync/async reset flows:

case THROW_SYNC_RESET:
        DEBUG_MSG( DEBUG_NAME , this, "throwing reset for");
	if ( m_reset_event_p ) m_reset_event_p->notify();
        throw sc_unwind_exception( this, true );
...
case THROW_KILL:
        DEBUG_MSG( DEBUG_NAME, this, "throwing kill for");
	throw sc_unwind_exception( this, false );

I am using MinGW GCC 7.4.0 with SEH exceptions. The throw statements up there end up calling RaiseException which is a Windows API that then causes the crash. I have spent days debugging that part and I was not able to figure it out.

I need to use SEH exceptions instead of SJLJ exceptions. Reason being, that SJLJ exceptions while it works with additions in sc_cor_qt similar to those in sc_cor_fibers, side effects exist such as having an incomplete backtrace in case we're in a try block.

 

So, I went ahead and created a commit here https://github.com/moayman/systemc/commit/e3790038752753bbbc2b0f30c064faae058ff227 that you may want to check out. A note about my commit:

• I started from SystemC 2.3.2 as I had a problem compiling 2.3.4 due to sc_mutex/sc_host_mutex

 

I would really appreciate any kind of help with this issue. We're so close to having Quickthreads support for 64 bit of Windows.

 

Thanks a lot.

 

[maehne]

Thanks @mo_ayman for all your effort in trying to fix QuickThreads on the 64-bit Windows platforms and sharing the current state of your fixes with us! Indeed, SystemC should default to use WinFibers on the Windows platform. Historically, this was only reflected in the MSVC project files, but not in the configure.ac files, as the latter would only work in Unix environments. Much is also handled in the source code itself using conditional compilation based on compiler/platform-specific preprocessor definitions. Cygwin as a POSIX environment for Windows was always a special case, which I got the impression was less popular among SystemC users on Windows. The same holds true for the much younger MSYS2 environment. Few of the developers, which contribute to the proof-of-concept implementation of SystemC are using it on Windows and then usually with MSVC. The last person, who had a look at this problem was @ralph.goergen back in 2016. Therefore, we are lacking experience with the particularities of the Cygwin and MSYS2 environments to support you in finding the cause for the remaining issue related to exception handling. As you may have noticed, the co-routines code is very old dating back to 2001. @Andy Goodrich, @Philipp A Hartmann, and @jeromecornetst did touch to some extend the concerned source files, so I don't know to which degree they may be able to help you.

Could you provide besides your patch to the SystemC implementation also a small test case illustrating your remaining issues? That may help to motivate someone else to have a look to the issue, too.

[maehne]

@Matthias Jung may be interested in your fixes, as he also stumbled on this issue. He also created a GitHub issue on the official public GitHub repository of the SystemC proof-of-concept implementation.

[mo_ayman]

@maehne,

First of all, I would like to thank you a lot for the kind words and the info you provided.

So, since my post I have been digging deep into the issue and reached the following:

• The issue stems from the use of exceptions from within a Quickthread
• I managed to create a minimal testcase that only includes Quickthreads code and a part of an example that they provided in meas.c. You can find that attached.
• I came across a couple of good reads and found that:
  • In case of SEH exceptions to be able to unwind a function unwind info must exist in the .pdata/.xdata sections of the binary
  • I also found that GCC provides some assembly pseudos that start with .seh_ that adds the necessary unwind info for assembly functions
    • The pseudos I am talking about are similar to those https://docs.microsoft.com/en-us/cpp/build/exception-handling-x64?view=vs-2019#unwind-helpers-for-masm
      • You can find the GCC ones at https://sourceware.org/legacy-ml/binutils/2009-08/msg00193.html
    • I added those and it failed as it used to do
    • I was able to verify that the unwind info was added for the qt_block function using dumpbin /unwindinfo

 

I really can't figure it out and I have lost hope to be honest, it has been quite a while that I have been trying to get this to work.

But, I believe it is either a GCC issue with SEH or we're hitting some kind of a limitation and what we need to do is not achievable given the way Quickthreads messes with the stack and the way Microsoft does things and all the security_cookie_checks that happen.

Either that or my code modifications to Quickthreads are bad, although I don't think so given that they work in call cases except for when exceptions are thrown.

I really am in a desperate need of some help.

sehissue.zip

[maehne]

Thanks @mo_ayman for all the additional information and minimal example demonstrating the problem! Have you tried to reach out to the developers of mingw-w64 and of MSYS2? Over the years, quite some issues were raised relating to exception handling for both projects. The minimal example, which you provide, should be useful to debug it on the mingw-w64 and MSYS2 side.

Indeed, QuickThreads is becoming a maintenance burden, as it is abandoned upstream for many years and few people are knowledgeable about its working. Porting it to new platforms has always been a headache. Maybe, this part of the SystemC can be reimplemented in some future version in a portable way using C++'20 coroutines once they become more widely available.

[mo_ayman]

Yes, I have actually created a post on the MinGW forum that can be found there: https://sourceforge.net/p/mingw-w64/discussion/723798/thread/7b5436a4d9/. I though got nothing.

Your suggestion about overhauling this part of SystemC sounds really good. The only concern for me is that C++20 is pretty new and it might require users to upgrade their compilers to provide C++20 support.