About N3 crashed

Our Project have 20 N3,10 CAB 8i ,50 ci20x8 amp.
Every 10 N3  one group, so we have two group, one main, another one backup.
We have 3 vlan for cobranet.

Our problem is:

When main group of a random N3 have HF2 error it bacome stop,then main of another random N3 will have follow error message. Please give me some direction to solve，Maybe where have prolem ?network?softwere? or other things.How to find it? Thank you.

1/15/2014 21:51:02    27440    note    piond/xdab/leader    arbitration done; ring is incomplete in redundant failed mode
1/15/2014 21:51:02    27439    note    piond/mute    unmuted
1/15/2014 21:51:02    27438    note    piond/xdab/leader    finaly unmuting after repeated failures
1/15/2014 21:50:54    27437    note    piond/xdab/leader    poll returned false: 'N006'
1/15/2014 21:50:40    27436    error    piond/xdab/leader    communication failure
1/15/2014 21:50:39    27435    error    piond/xdab/leader    exception: N001:0xd2915f55: '/dev/pion/sharc2: software timeout on command operation'
1/15/2014 21:50:34    27434    note    piond/mute    muted: xdab reconfiguration
1/15/2014 21:49:43    27433    note    piond/xdab/leader    arbitration done; ring is incomplete in redundant failed mode
1/15/2014 21:49:43    27432    note    piond/mute    unmuted
1/15/2014 21:49:43    27431    note    piond/xdab/leader    finaly unmuting after repeated failures
1/15/2014 21:49:33    27430    error    piond/xdab/leader    exception: N004:0x366b3eb2: '/dev/pion/sharc2: software timeout on command operation'
1/15/2014 21:49:29    27429    error    piond/xdab/leader    communication failure
1/15/2014 21:49:22    27428    note    piond/mute    muted: xdab reconfiguration
1/15/2014 21:49:13    27427    note    piond/xdab/leader    arbitration done; ring is incomplete in redundant failed mode
1/15/2014 21:49:13    27426    note    piond/mute    unmuted
1/15/2014 21:48:59    27425    note    piond/role_manager    role is running
1/15/2014 21:48:53    27424    note    project    cab 0xc202 connected
1/15/2014 21:48:53    27423    note    project    cab 0x0 changed ID to 0xc202
1/15/2014 21:48:47    27422    note    project    user logged on: defaultuser
1/15/2014 21:48:47    27421    note    project    user logged on: defaultuser
1/15/2014 21:48:01    27420    note    piond/role_manager    restarting role : Audio-V1.6.1-10Z-final/N008/deAcxzjm9icPmozZCEA-aOX4BKz/qg7JPZPVkSM9yNS0BOgrd4Ty-yv
1/15/2014 21:48:00    27419    fault    piond/pacman    real-time thread error
1/15/2014 21:48:00    27418    error    piond/dsp    30 sequential frames with dma errors - aborting
1/15/2014 21:48:00    27417    note    project    user logged off: defaultuser
1/15/2014 21:48:00    27416    error    piond/fault_policy    restarting audio engine
1/15/2014 21:48:00    27415    fault    piond/role_manager    role fault
1/15/2014 21:48:00    27414    note    project    user logged off: defaultuser
1/15/2014 21:48:00    27413    note    project    user logged off: defaultuser
1/15/2014 21:48:00    27412    fault    piond/sound_engine    exception during poll: /dev/pion/sharc2: software timeout on command operation
1/15/2014 21:47:59    27411    error    piond/dsp    5 sequential frames with dma errors
1/15/2014 21:47:56    27410    note    piond/mute    muted: xdab reconfiguration
1/15/2014 21:47:51    27409    error    piond/xdab/leader    communication failure
1/15/2014 21:47:51    27408    note    piond/xdab/leader    poll returned false: 'N007'