Jump to content

Recommended Posts

@David BlackWhat would you say is standard praxis for parsing logs to figure out pass/fail status? Find standard patterns for UVM errors + find simulator specific patterns for other errors + make sure there is a UVM report to catch silent errors stopping the simulation?

Do people ever look at the return code? Testing at EDA Playground I see that some simulators return a non-zero code for some, but not all, errors. Why isn't this used more? Keeps me from knowing simulator specific error messages and makes the script more portable.

Share this post


Link to post
Share on other sites

The return code usually indicates successful completion of the tool and is unrelated to completion of the test. Non-zero return codes would be OS specific error codes. 

The SystemVerilog standard way of indicating pass/fail status is using the $info/$warning/$error/$fatal messages. Most tools are essentially catch the UVM reports and convert them to one of the SystemVerilog messages. They also have a way of detecting the most severe message issued during a run and you can use that information for determining pass/fail status. 

Share this post


Link to post
Share on other sites

Thanks @dave_59. I did a quick test with the commercial simulators available at EDA playground with the following error scenarios:

1. Assert with $error
2. Null object dereferencing

Riviera-PRO would do a normal exit in both these scenarios, Cadence has a non-zero return code for both and Synopsis return non-zero for the null object dereferencing but not for the assert. I didn't check if there are option flags that would change this behavior but it seems that there are different opinions on how the return code should be used.

Personally I would fail a test with a non-zero exit regardless of simulator strategy and log file contents.

I would also prefer that the tools would use the return code for the errors they know about, at least as an option. Let's say I have a smaller project just using SystemVerilog with $error or VHDL asserts with error severity. The return code would give me pass/fail status if the assertions fail or if there is another problem like null object dereferencing . If it passed I don't care about the logs and if it failed they are small enough to be read manually.

If I have a project using less specific error mechanisms like Verilog $display I would need parsing but the scope of parsing is reduced.

If I have many long log files I may need scripting to mine them for the interesting events but in that case I rather have more machine readable formats that is well supported by scripting languages. XML or JSON for example. It would make the scripting easier and less error prone.

Share this post


Link to post
Share on other sites

Just a few short thoughts:

1. If using SystemC or UVM, you should use restrict messages to the respective SC_REPORT_* or `uvm_report_* macros. These have standardized output formats, which makes them easier to use.

2. If using SystemVerilog without UVM, then use $info, $error, etc. for similar reasons

3. At the end of a simulation, you can detect how many errors etc. and use a return code. If would also suggest always printing a summary of the run (e.g. how many warnings and erros), and ALWAYS show one of two messages at the end: PASSED or FAILED.

Makes life easier. My reason for the original logscan tool was situations where you don't have control over the error messages (e.g. certain EDA tools output).

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×