multiple threads calling sequencer lock results in deadlock

[Lawrence Said]

In my testbench I have two background sequences (call them S1 and S2) perform register access and a test that has multiple threads ( call them T1 to T7) also performing register access on the same shared bus.  T1 through T7 need to lock the sequencer in order to perform multi-cycle transactions on the CPU bus. The scenario is

1. S1 issues a write, resulting in arb_sequence_q.size == 1 and arb_sequence_q[0].request == SEQ_TYPE_REQ
2. in the same simulation tick, T1 through T7 issue lock requests. arb_sequence_q.size() == 8 and arb_sequence_q[0] is still the S1 request
3. S2 issues a read, resulting in arb_sequence_q.size() == 9, arb_sequence_q[8].request == SEQ_TYPE_REQ
4. The CPU driver calls get() and the request at the head of arb_sequence_q (the write from S1) gets popped, resulting in arb_sequence_q.size() == 8 and arb_sequence_q[0:6].request = SEQ_TYPE_LOCK and arb_sequence_q[7].request == SEQ_TYPE_REQ (this is the request from S2). 
5. finish_item is called for the S1 request, resulting in grant_queued_locks getting called, pushing all LOCK requests onto the lock_list and leaving only S2 on the arb_sequence_q. 
   1. arb_sequence_q.size() == 1, arb_sequence_q[0].request = SEQ_TYPE_REQ (this is the S2 request)
   2. lock_list.size() == 7
   3. arb_completed.num() == 0
6. T1 calls write, enqueuing a SEQ_TYPE_REQ on the arb_sequence_q. The arb_sequence_q.size() == 2, arb_sequence_q[0] is S2's request and arb_sequence_q[1] is T1's request.
7. [edit] Here's where the problem hits. Because the lock_list size() > 1, when the consumer (driver) calls get(), the sequencer will cycle through all its requests in its arb_sequence_q until it finds one that is ~is_blocked(). Unfortunately, that function cycles through all sequences in the lock_list regardless of whether the first item in the lock list corresponds to the request being processed. So since we have 7 locks in there, one for each thread, when the driver tries to get a new sequence, T1 will always be blocked by T2:T7. Similarly any request from T1:T7 will get locked out by the others. 

We've now deadlocked. 

I believe there is an issue here with the is_blocked() function which should have some form of priority encoding.

[Lawrence Said]

bump

[Lawrence Said]

anyone else run into this?