I stink at writing test plans.

[c4brian]

Every time I sit down to write down my test plan, I simply feel as though I'm regurgitating my design requirements.  I've been through the Mentor Coverage Cookbook a few times already;  It has a UART and a couple other things.  Does anyone know other good examples / explanations of test plans? 

Everybody writes one of these things... right?

 

My example "function":

Receive an ARP request; generate an ARP reply.  The destination IP of the ARP must match the DUT IP in order for DUT to generate reply.

 

What line item(s) would I write in my "test plan"?

 

ARP request received by DUT    Type: covergroup, cross                     

ARP reply sent by DUT               Type: covergroup, cross

 

This is where I get really wrapped around the axle.  I already know what I sit down to write the test, I am going to simply randomize an ARP packet, and bombard my DUT.  It will receive ARP Replies (which it should drop), ARP Request addressed to someone else (drop this as well), and occasionally a Request with the correct address (accept and reply).

 

I definitely want coverpoints on all the non-supported behavior as well, but I take it that's not supposed to be mentioned in the test plan anywhere?   The whole point of constrained random stimulus generation is to find the bugs you aren't looking for, and it must be paired with a coverage collector to tell you where your blind sheep has wandered.

[tudor.timi]

I don't really remember what the Verification Academy cookbook said (it's a long time since I've seen it). I can tell you how I've started writing my plans. For your small little feature, I'd structure it like this:

ARP interface

-- The DUT should reply to an ARP request addressed to it
---- <map some cover item where I show I did ARP requests to the DUT>

-- The DUT should drop ARP requests not addressed to it
---- <map some cover item where I show I did ARP requests to some other device>

-- The DUT should drop ARP replies
---- <map some cover item where I know I did ARP replies>

The "--" notation above means that it's a child section. I'd have the one big section related to ARP processing, then split this into smaller sub-sections, one per requirement. I don't get what you mean by non-supported behavior. If you're referring to dropping replies and other requests, that's not non-supported behavior, just other requirements aside from the main one, that is should accept ARP requests addressed to it.

 

I'd typically break down a specification like this to end up with a verification plan. That doesn't mean what I think of at that point is the final version. W.r.t. finding the bugs you aren't looking for, you're right, that's a big part of constrained random (the other being test automation). What I do when I find a bug in an area I wasn't expecting (i.e. I didn't have any coverage defined for) is to add some coverage in the general area to make sure I stress it enough and the bug doesn't reappear. Other times you just get ideas of more stuff you want to test out of the blue. A verification plan should be a living thing, like your testbench and tests.

[c4brian]

Yes; great input.  What I mean about non-supported functionality is the myriad of input packet types that I don't concern myself with; for example, I accept only a few ICMP types, but several I don't.  I think I'll identify this by specifying something like "shall only accept a supported ICMP type" (specify these) , then cover all the other types through randomization (and some invalid ICMP types).

[c4brian]

Thought of a good question; in regard to:

 

-- The DUT should reply to an ARP request addressed to it
---- <map some cover item where I show I did ARP requests to the DUT>

 

I have a coverage item identifying that I issued an ARP request; do I need I need to also sample/cover the ARP reply as well?  Or is that not necessary, since my scoreboard checks that it was, in fact, generated?

[tudor.timi]

Depends on how paranoid you are. It won't hurt to cover the reply as well, but my view is that you generally want to cover stimulus that you drive into the design. That's not a hard rule, though, since sometimes you want to cover that you reached certain states, for example, regardless of how you got there.

[c4brian]

This was the direction I was leaning as well.  Excellent.