XML output and SQL databases for debug

[nbjessen]

Debug is extremely difficult to do in large, constrained-random simulations.

1) Even if there is large amounts of data available (such as UVM seq-item dumps), it is extremely slow to analyze the standard text outputs, and requires lots of script writing for each new bug

2) There are times when multiple streams of different sequence-items must be parsed in order to put the suspect sequence-item into context.

 

I would like to propose adding another UVM output format: XML

 

Then standard XML report-writers could be used to analyze the multiple output streams and generate statistics or identify possible failures or the failures themselves.

 

I would then to also like to propose loading that data into SQL databases for more complicated analysis and creation of standard reports.

 

Erik Jessen

[David Black]

Interesting. I created my own reporting scheme some years back under SystemC with XML output as an alternative. It was part of my revamped report_handler. It should also be noted that transaction recording should offer a similar offering so that reporting and recording can be merged. This could lead to some very powerful debug tools.

 

Of course a carefully thought out XML schema should accompany such a proposal, and perhaps an option to have things like sprint optionally output XML for transactions.

 

As for SQL, I think that could be separated out and left for tools implementations.

[nbjessen]

David - yes, I was envisioning a single standard output for all transactions/sequence-items, so that things could be compared.

Ideally we would have standard, reusable classes for various interface standards (XGMII, SPI, PCIe, etc.)
 

Here's why I would like to use SQL up-front (if only for planning): it will force users to decide how they can track traffic - what will they use for keys to uniquely identify each transaction?

I have observed lots of cases where one simply can't tell which output was generated by which input - there's just no way to tell.  And in one case, it turned out that we had false-passes; the input just *happened* to align the stars so that two outputs looked like a pass, but really its because we had double-bugs cancelling each other out.  It wasn't until system sim that we found we had inputs -> outputs swapping around in a corner case.

If we'd changed the RTL to having a tracking tag, we could have easily caught the problem much earlier.  And of course if this went into production it'd have been almost impossible to debug.

 

I also like the SQL upfront - because it's a LOT easier upfront to ensure all that data can be tracked, than to go back later and realize that you could track the data in *almost* all of the time.  But that *almost* meant that you had to do either incredibly complex analysis to get to 100%-sure, or you had to do 'best guess' after a lot of analysis.  Both are extremely costly for what's really a trivial amount of upfront work.

[uwes]

hi,

 

> it is extremely slow to analyze the standard text outputs, and requires lots of script writing for each new bug

 

isnt this not the issue to begin with? all commercial eda tools have reasonable debug,tracing,breakpoints/watchpoints/etc.. you have transaction databases, special trace knobs,  etc and even things like reverse stepping. logfile debugging doesnt seem right in a modern world.

 

technically you could record in XML or SQL via a specialized uvm_tr_database backend but whatever you do you also need the rest of the analysis system to find what you want. oh and btw. pushing data into SQL isnt much slower than the signal/transaction recording typically used/optimized in each simulator.

 

/uwe

[nbjessen]

Uwe, my goal is to only do the post-processing if a failure occurs, so that the simulator licenses are available as much as possible.

 

Also, we have a highly parallel processing architecture, so the problem is not "did we get an unexpected output", but being able to track backwards to watch the data propagate through the design.

 

Also - while some corporate cultures are quite advanced on their uses of debug techniques and tools, not all are. Also there are times when the vendor tools are not stable.

In my particular case, I pretty much could guarantee a core-dump if I did anything other than waveform dump and text processing - I was never able to identify in my very large design exactly what caused even turning on break points (but not even using them) to cause everything to crash.  Even in post-processing, just source-code browsing caused the entire simulator GUI to crash.

 

Also, in my case each sequence-item has 2 or 3 levels of linked-lists of objects in them, and some words have up to 8 possible interpretations.

 

I can't even use the `uvm_fields for debug because the simulator crashes on me, reporting that I've exceeded the string-expansion limit.

 

So I have both problems with UVM and with vendor tools.  But I trust text output.  And one the text output is in a standard format, I have a lot more options for post-processing.

Erik

[uwes]

hi,

 

basically you confirm my suspicion that the real issue you got is somewhere else. what you are asking for is a 'workaround' (which in itself requires infrastructure/tools which in turn have to be written/maintained/...)

 

/uwe

[nbjessen]

Uwe,

You are quite sure that you know what I am saying and what technical problems I'm dealing with.

However, I'm quite sure that you don't.

 

So - are you open to the possibility that you don't yet understand what I am saying (and we should keep talking), or should we just end the discussion?

 

Erik

[uwes]

erik,

 

i'm hearing two things:

 

1. you cant do reasonable "first level debug" (that's why you do text/log debug and your tool/gui is crashing). this is (i hope you agree) a vendor issue

2. you are asking for additional interfaces to perform debug on an "application level" partially because of #1 and you got custom/high level things you want automated analysis on

 

i hope this captures your issues. now #1 is something the vendor has to fix so that you dont need text debug/low level debug in text files.  if #1 is fixed you can focus upon the application/custom level debug for which an transaction level interface to a "db" makes sense. I also understand that some people like/do logfile debug because its reliable.

 

a few suggestions 

 

1. you can build today a uvm_report_server which emits messages in xml format. you can then extract the messages from the log

2. you can have the report-server of #1 directly pushing the message into a custom db (sql, log4*,... via dpi)

3. you can record (uvm11) you transactions into a vendor db using uvm transaction recording. the vendor db can then be analyzed with vendor tools (TXE in cadence)

4. you also get transaction recording into a parsable text format in uvm11

5. with uvm12 you can redirect ALL uvm messages to the recording backend. that lets you record all transaction+messages into a custom/vendor database

6. there is a white paper on cadence support regarding an UVM/sqlite adapter to record arbitrary transactions. goto support.cadence.com and search for "uvm" + "sql"

7. i do not recall real discussions to supply specialized recording outputs for xml/sql in the viptsc. also that would always carry the question of a common schema ...

8. you might want to explore alternate debug paths such as http://www.cadence.com/products/fv/debug_analyzer/pages/default.aspx

 

hope that helps

/uwe

[nbjessen]

Uwe,

Many thanks!

#1: already working on it.

#2: this is my plan once #1 is done.

#3: We have multiple simulator licences and multiple vendor hardware accelerators.  So a vendor-specific DB won't work.

#4: this was discarded as the data-structures in the transactions are highly variable - much easier to output XML

#5: Haven't looked at UVM1.2 yet, as company is still using UVM 1.0.  Thanks for the tip!

#6: thanks - I'll go take a look, but because we use multiple simulators, I'd need something that works on all simulators.

#7: Yes, that is one of the things I've been mapping out; I've downloaded the OCP/IP to see what they have done for vendor-specific IP-XACT schema for busses.

#8: Again - we use multiple simulators.

 

Erik