Jump to content

Log File Parsing


Recommended Posts

@David BlackWhat would you say is standard praxis for parsing logs to figure out pass/fail status? Find standard patterns for UVM errors + find simulator specific patterns for other errors + make sure there is a UVM report to catch silent errors stopping the simulation?

Do people ever look at the return code? Testing at EDA Playground I see that some simulators return a non-zero code for some, but not all, errors. Why isn't this used more? Keeps me from knowing simulator specific error messages and makes the script more portable.

Link to comment
Share on other sites

The return code usually indicates successful completion of the tool and is unrelated to completion of the test. Non-zero return codes would be OS specific error codes. 

The SystemVerilog standard way of indicating pass/fail status is using the $info/$warning/$error/$fatal messages. Most tools are essentially catch the UVM reports and convert them to one of the SystemVerilog messages. They also have a way of detecting the most severe message issued during a run and you can use that information for determining pass/fail status. 

Link to comment
Share on other sites

Thanks @dave_59. I did a quick test with the commercial simulators available at EDA playground with the following error scenarios:

1. Assert with $error
2. Null object dereferencing

Riviera-PRO would do a normal exit in both these scenarios, Cadence has a non-zero return code for both and Synopsis return non-zero for the null object dereferencing but not for the assert. I didn't check if there are option flags that would change this behavior but it seems that there are different opinions on how the return code should be used.

Personally I would fail a test with a non-zero exit regardless of simulator strategy and log file contents.

I would also prefer that the tools would use the return code for the errors they know about, at least as an option. Let's say I have a smaller project just using SystemVerilog with $error or VHDL asserts with error severity. The return code would give me pass/fail status if the assertions fail or if there is another problem like null object dereferencing . If it passed I don't care about the logs and if it failed they are small enough to be read manually.

If I have a project using less specific error mechanisms like Verilog $display I would need parsing but the scope of parsing is reduced.

If I have many long log files I may need scripting to mine them for the interesting events but in that case I rather have more machine readable formats that is well supported by scripting languages. XML or JSON for example. It would make the scripting easier and less error prone.

Link to comment
Share on other sites

Just a few short thoughts:

1. If using SystemC or UVM, you should use restrict messages to the respective SC_REPORT_* or `uvm_report_* macros. These have standardized output formats, which makes them easier to use.

2. If using SystemVerilog without UVM, then use $info, $error, etc. for similar reasons

3. At the end of a simulation, you can detect how many errors etc. and use a return code. If would also suggest always printing a summary of the run (e.g. how many warnings and erros), and ALWAYS show one of two messages at the end: PASSED or FAILED.

Makes life easier. My reason for the original logscan tool was situations where you don't have control over the error messages (e.g. certain EDA tools output).

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Create New...