Untitled

 avatar
unknown
batchfile
a year ago
47 kB
17
Indexable
===================================================================================================================
► 每個『組合實例』(t1 fused vs. t2 separate)

[1] sub_fusion            │ /embeddings/LayerNorm/ReduceMean
  t1 fused    : 0.003200 ms
  t2 separate : 0.004256 ms  (Δ 0.001056 ms, speed-up 1.33×)
  ── t2 breakdown ──
    normal         : 0.002208 ms
    Sub            : 0.002048 ms

[2] normalization_fusion  │ /embeddings/LayerNorm/ReduceMean_1
  t1 fused    : 0.004065 ms
  t2 separate : 0.011074 ms  (Δ 0.007009 ms, speed-up 2.72×)
  ── t2 breakdown ──
    normal         : 0.002016 ms
    Add            : 0.001153 ms
    Sqrt           : 0.000928 ms
    Div            : 0.002368 ms
    Mul            : 0.002241 ms
    Add            : 0.002368 ms

[3] Add                   │ /encoder/layer.0/attention/self/query/MatMul
  t1 fused    : 0.023361 ms
  t2 separate : 0.025185 ms  (Δ 0.001824 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022881 ms
    Add            : 0.002304 ms

[4] Add                   │ /encoder/layer.0/attention/self/key/MatMul
  t1 fused    : 0.023361 ms
  t2 separate : 0.025281 ms  (Δ 0.001920 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022913 ms
    Add            : 0.002368 ms

[5] Add                   │ /encoder/layer.0/attention/self/value/MatMul
  t1 fused    : 0.023425 ms
  t2 separate : 0.025185 ms  (Δ 0.001760 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022849 ms
    Add            : 0.002336 ms

[6] div+add               │ /encoder/layer.0/attention/self/MatMul
  t1 fused    : 0.005568 ms
  t2 separate : 0.007104 ms  (Δ 0.001536 ms, speed-up 1.28×)
  ── t2 breakdown ──
    normal         : 0.004640 ms
    Div            : 0.002464 ms

[7] double_add            │ /encoder/layer.0/attention/output/dense/MatMul
  t1 fused    : 0.023712 ms
  t2 separate : 0.026402 ms  (Δ 0.002690 ms, speed-up 1.11×)
  ── t2 breakdown ──
    normal         : 0.022881 ms
    Add            : 0.002304 ms
    Add            : 0.001217 ms

[8] sub_fusion            │ /encoder/layer.0/attention/output/LayerNorm/ReduceMean
  t1 fused    : 0.003104 ms
  t2 separate : 0.004032 ms  (Δ 0.000928 ms, speed-up 1.30×)
  ── t2 breakdown ──
    normal         : 0.002112 ms
    Sub            : 0.001920 ms

[9] normalization_fusion  │ /encoder/layer.0/attention/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.004032 ms
  t2 separate : 0.011168 ms  (Δ 0.007136 ms, speed-up 2.77×)
  ── t2 breakdown ──
    normal         : 0.002048 ms
    Add            : 0.001184 ms
    Sqrt           : 0.000928 ms
    Div            : 0.002304 ms
    Mul            : 0.002368 ms
    Add            : 0.002336 ms

[10] auto_fusion           │ /encoder/layer.0/intermediate/dense/MatMul
  t1 fused    : 0.034786 ms
  t2 separate : 0.040769 ms  (Δ 0.005983 ms, speed-up 1.17×)
  ── t2 breakdown ──
    normal         : 0.023649 ms
    Add            : 0.003776 ms
    Div            : 0.003808 ms
    Erf            : 0.002048 ms
    Add            : 0.002816 ms
    Mul            : 0.001824 ms
    Mul            : 0.002848 ms

[11] double_add            │ /encoder/layer.0/output/dense/MatMul
  t1 fused    : 0.083362 ms
  t2 separate : 0.086146 ms  (Δ 0.002784 ms, speed-up 1.03×)
  ── t2 breakdown ──
    normal         : 0.082722 ms
    Add            : 0.002208 ms
    Add            : 0.001216 ms

[12] sub_fusion            │ /encoder/layer.0/output/LayerNorm/ReduceMean
  t1 fused    : 0.003104 ms
  t2 separate : 0.004064 ms  (Δ 0.000960 ms, speed-up 1.31×)
  ── t2 breakdown ──
    normal         : 0.002112 ms
    Sub            : 0.001952 ms

[13] normalization_fusion  │ /encoder/layer.0/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.004032 ms
  t2 separate : 0.011104 ms  (Δ 0.007072 ms, speed-up 2.75×)
  ── t2 breakdown ──
    normal         : 0.002144 ms
    Add            : 0.001152 ms
    Sqrt           : 0.000896 ms
    Div            : 0.002336 ms
    Mul            : 0.002240 ms
    Add            : 0.002336 ms

[14] Add                   │ /encoder/layer.1/attention/self/query/MatMul
  t1 fused    : 0.023329 ms
  t2 separate : 0.025153 ms  (Δ 0.001824 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022881 ms
    Add            : 0.002272 ms

[15] Add                   │ /encoder/layer.1/attention/self/key/MatMul
  t1 fused    : 0.023424 ms
  t2 separate : 0.025249 ms  (Δ 0.001825 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022913 ms
    Add            : 0.002336 ms

[16] Add                   │ /encoder/layer.1/attention/self/value/MatMul
  t1 fused    : 0.023361 ms
  t2 separate : 0.025153 ms  (Δ 0.001792 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022880 ms
    Add            : 0.002273 ms

[17] div+add               │ /encoder/layer.1/attention/self/MatMul
  t1 fused    : 0.005601 ms
  t2 separate : 0.007008 ms  (Δ 0.001407 ms, speed-up 1.25×)
  ── t2 breakdown ──
    normal         : 0.004544 ms
    Div            : 0.002464 ms

[18] double_add            │ /encoder/layer.1/attention/output/dense/MatMul
  t1 fused    : 0.023649 ms
  t2 separate : 0.026402 ms  (Δ 0.002753 ms, speed-up 1.12×)
  ── t2 breakdown ──
    normal         : 0.022881 ms
    Add            : 0.002305 ms
    Add            : 0.001216 ms

[19] sub_fusion            │ /encoder/layer.1/attention/output/LayerNorm/ReduceMean
  t1 fused    : 0.003073 ms
  t2 separate : 0.004097 ms  (Δ 0.001024 ms, speed-up 1.33×)
  ── t2 breakdown ──
    normal         : 0.002144 ms
    Sub            : 0.001953 ms

[20] normalization_fusion  │ /encoder/layer.1/attention/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.004000 ms
  t2 separate : 0.011008 ms  (Δ 0.007008 ms, speed-up 2.75×)
  ── t2 breakdown ──
    normal         : 0.002048 ms
    Add            : 0.001152 ms
    Sqrt           : 0.000896 ms
    Div            : 0.002336 ms
    Mul            : 0.002304 ms
    Add            : 0.002272 ms

[21] auto_fusion           │ /encoder/layer.1/intermediate/dense/MatMul
  t1 fused    : 0.034625 ms
  t2 separate : 0.040994 ms  (Δ 0.006369 ms, speed-up 1.18×)
  ── t2 breakdown ──
    normal         : 0.023937 ms
    Add            : 0.003680 ms
    Div            : 0.003841 ms
    Erf            : 0.002048 ms
    Add            : 0.002816 ms
    Mul            : 0.001856 ms
    Mul            : 0.002816 ms

[22] double_add            │ /encoder/layer.1/output/dense/MatMul
  t1 fused    : 0.083554 ms
  t2 separate : 0.086306 ms  (Δ 0.002752 ms, speed-up 1.03×)
  ── t2 breakdown ──
    normal         : 0.082818 ms
    Add            : 0.002272 ms
    Add            : 0.001216 ms

[23] sub_fusion            │ /encoder/layer.1/output/LayerNorm/ReduceMean
  t1 fused    : 0.003104 ms
  t2 separate : 0.004096 ms  (Δ 0.000992 ms, speed-up 1.32×)
  ── t2 breakdown ──
    normal         : 0.002176 ms
    Sub            : 0.001920 ms

[24] normalization_fusion  │ /encoder/layer.1/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.004000 ms
  t2 separate : 0.011041 ms  (Δ 0.007041 ms, speed-up 2.76×)
  ── t2 breakdown ──
    normal         : 0.002112 ms
    Add            : 0.001152 ms
    Sqrt           : 0.000896 ms
    Div            : 0.002304 ms
    Mul            : 0.002304 ms
    Add            : 0.002273 ms

[25] Add                   │ /encoder/layer.2/attention/self/query/MatMul
  t1 fused    : 0.023552 ms
  t2 separate : 0.025664 ms  (Δ 0.002112 ms, speed-up 1.09×)
  ── t2 breakdown ──
    normal         : 0.023296 ms
    Add            : 0.002368 ms

[26] Add                   │ /encoder/layer.2/attention/self/key/MatMul
  t1 fused    : 0.023424 ms
  t2 separate : 0.025313 ms  (Δ 0.001889 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022881 ms
    Add            : 0.002432 ms

[27] Add                   │ /encoder/layer.2/attention/self/value/MatMul
  t1 fused    : 0.023393 ms
  t2 separate : 0.025153 ms  (Δ 0.001760 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022849 ms
    Add            : 0.002304 ms

[28] div+add               │ /encoder/layer.2/attention/self/MatMul
  t1 fused    : 0.005568 ms
  t2 separate : 0.007041 ms  (Δ 0.001473 ms, speed-up 1.26×)
  ── t2 breakdown ──
    normal         : 0.004545 ms
    Div            : 0.002496 ms

[29] double_add            │ /encoder/layer.2/attention/output/dense/MatMul
  t1 fused    : 0.023649 ms
  t2 separate : 0.026465 ms  (Δ 0.002816 ms, speed-up 1.12×)
  ── t2 breakdown ──
    normal         : 0.022913 ms
    Add            : 0.002304 ms
    Add            : 0.001248 ms

[30] sub_fusion            │ /encoder/layer.2/attention/output/LayerNorm/ReduceMean
  t1 fused    : 0.003072 ms
  t2 separate : 0.004031 ms  (Δ 0.000959 ms, speed-up 1.31×)
  ── t2 breakdown ──
    normal         : 0.002111 ms
    Sub            : 0.001920 ms

[31] normalization_fusion  │ /encoder/layer.2/attention/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.004000 ms
  t2 separate : 0.011168 ms  (Δ 0.007168 ms, speed-up 2.79×)
  ── t2 breakdown ──
    normal         : 0.002144 ms
    Add            : 0.001152 ms
    Sqrt           : 0.000928 ms
    Div            : 0.002336 ms
    Mul            : 0.002304 ms
    Add            : 0.002304 ms

[32] auto_fusion           │ /encoder/layer.2/intermediate/dense/MatMul
  t1 fused    : 0.034753 ms
  t2 separate : 0.041345 ms  (Δ 0.006592 ms, speed-up 1.19×)
  ── t2 breakdown ──
    normal         : 0.024033 ms
    Add            : 0.003936 ms
    Div            : 0.003808 ms
    Erf            : 0.002048 ms
    Add            : 0.002848 ms
    Mul            : 0.001856 ms
    Mul            : 0.002816 ms

[33] double_add            │ /encoder/layer.2/output/dense/MatMul
  t1 fused    : 0.083426 ms
  t2 separate : 0.086211 ms  (Δ 0.002785 ms, speed-up 1.03×)
  ── t2 breakdown ──
    normal         : 0.082595 ms
    Add            : 0.002400 ms
    Add            : 0.001216 ms

[34] sub_fusion            │ /encoder/layer.2/output/LayerNorm/ReduceMean
  t1 fused    : 0.003105 ms
  t2 separate : 0.004128 ms  (Δ 0.001023 ms, speed-up 1.33×)
  ── t2 breakdown ──
    normal         : 0.002208 ms
    Sub            : 0.001920 ms

[35] normalization_fusion  │ /encoder/layer.2/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.003999 ms
  t2 separate : 0.011265 ms  (Δ 0.007266 ms, speed-up 2.82×)
  ── t2 breakdown ──
    normal         : 0.002145 ms
    Add            : 0.001152 ms
    Sqrt           : 0.000896 ms
    Div            : 0.002336 ms
    Mul            : 0.002368 ms
    Add            : 0.002368 ms

[36] Add                   │ /encoder/layer.3/attention/self/query/MatMul
  t1 fused    : 0.023393 ms
  t2 separate : 0.025153 ms  (Δ 0.001760 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022881 ms
    Add            : 0.002272 ms

[37] Add                   │ /encoder/layer.3/attention/self/key/MatMul
  t1 fused    : 0.023488 ms
  t2 separate : 0.025281 ms  (Δ 0.001793 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022913 ms
    Add            : 0.002368 ms

[38] Add                   │ /encoder/layer.3/attention/self/value/MatMul
  t1 fused    : 0.023520 ms
  t2 separate : 0.025281 ms  (Δ 0.001761 ms, speed-up 1.07×)
  ── t2 breakdown ──
    normal         : 0.022880 ms
    Add            : 0.002401 ms

[39] div+add               │ /encoder/layer.3/attention/self/MatMul
  t1 fused    : 0.005568 ms
  t2 separate : 0.007040 ms  (Δ 0.001472 ms, speed-up 1.26×)
  ── t2 breakdown ──
    normal         : 0.004608 ms
    Div            : 0.002432 ms

[40] double_add            │ /encoder/layer.3/attention/output/dense/MatMul
  t1 fused    : 0.023712 ms
  t2 separate : 0.026369 ms  (Δ 0.002657 ms, speed-up 1.11×)
  ── t2 breakdown ──
    normal         : 0.022881 ms
    Add            : 0.002304 ms
    Add            : 0.001184 ms

[41] sub_fusion            │ /encoder/layer.3/attention/output/LayerNorm/ReduceMean
  t1 fused    : 0.003072 ms
  t2 separate : 0.004096 ms  (Δ 0.001024 ms, speed-up 1.33×)
  ── t2 breakdown ──
    normal         : 0.002144 ms
    Sub            : 0.001952 ms

[42] normalization_fusion  │ /encoder/layer.3/attention/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.004000 ms
  t2 separate : 0.011170 ms  (Δ 0.007170 ms, speed-up 2.79×)
  ── t2 breakdown ──
    normal         : 0.002113 ms
    Add            : 0.001152 ms
    Sqrt           : 0.000928 ms
    Div            : 0.002304 ms
    Mul            : 0.002305 ms
    Add            : 0.002368 ms

[43] auto_fusion           │ /encoder/layer.3/intermediate/dense/MatMul
  t1 fused    : 0.034625 ms
  t2 separate : 0.041121 ms  (Δ 0.006496 ms, speed-up 1.19×)
  ── t2 breakdown ──
    normal         : 0.023969 ms
    Add            : 0.003776 ms
    Div            : 0.003840 ms
    Erf            : 0.002048 ms
    Add            : 0.002816 ms
    Mul            : 0.001824 ms
    Mul            : 0.002848 ms

[44] double_add            │ /encoder/layer.3/output/dense/MatMul
  t1 fused    : 0.083394 ms
  t2 separate : 0.086178 ms  (Δ 0.002784 ms, speed-up 1.03×)
  ── t2 breakdown ──
    normal         : 0.082690 ms
    Add            : 0.002272 ms
    Add            : 0.001216 ms

[45] sub_fusion            │ /encoder/layer.3/output/LayerNorm/ReduceMean
  t1 fused    : 0.003104 ms
  t2 separate : 0.004128 ms  (Δ 0.001024 ms, speed-up 1.33×)
  ── t2 breakdown ──
    normal         : 0.002208 ms
    Sub            : 0.001920 ms

[46] normalization_fusion  │ /encoder/layer.3/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.004032 ms
  t2 separate : 0.011073 ms  (Δ 0.007041 ms, speed-up 2.75×)
  ── t2 breakdown ──
    normal         : 0.002112 ms
    Add            : 0.001152 ms
    Sqrt           : 0.000896 ms
    Div            : 0.002304 ms
    Mul            : 0.002305 ms
    Add            : 0.002304 ms

[47] Add                   │ /encoder/layer.4/attention/self/query/MatMul
  t1 fused    : 0.023392 ms
  t2 separate : 0.025153 ms  (Δ 0.001761 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022849 ms
    Add            : 0.002304 ms

[48] Add                   │ /encoder/layer.4/attention/self/key/MatMul
  t1 fused    : 0.023456 ms
  t2 separate : 0.025152 ms  (Δ 0.001696 ms, speed-up 1.07×)
  ── t2 breakdown ──
    normal         : 0.022848 ms
    Add            : 0.002304 ms

[49] Add                   │ /encoder/layer.4/attention/self/value/MatMul
  t1 fused    : 0.023457 ms
  t2 separate : 0.025152 ms  (Δ 0.001695 ms, speed-up 1.07×)
  ── t2 breakdown ──
    normal         : 0.022912 ms
    Add            : 0.002240 ms

[50] div+add               │ /encoder/layer.4/attention/self/MatMul
  t1 fused    : 0.005568 ms
  t2 separate : 0.007040 ms  (Δ 0.001472 ms, speed-up 1.26×)
  ── t2 breakdown ──
    normal         : 0.004544 ms
    Div            : 0.002496 ms

[51] double_add            │ /encoder/layer.4/attention/output/dense/MatMul
  t1 fused    : 0.023841 ms
  t2 separate : 0.026433 ms  (Δ 0.002592 ms, speed-up 1.11×)
  ── t2 breakdown ──
    normal         : 0.022881 ms
    Add            : 0.002336 ms
    Add            : 0.001216 ms

[52] sub_fusion            │ /encoder/layer.4/attention/output/LayerNorm/ReduceMean
  t1 fused    : 0.003104 ms
  t2 separate : 0.004096 ms  (Δ 0.000992 ms, speed-up 1.32×)
  ── t2 breakdown ──
    normal         : 0.002144 ms
    Sub            : 0.001952 ms

[53] normalization_fusion  │ /encoder/layer.4/attention/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.004032 ms
  t2 separate : 0.011201 ms  (Δ 0.007169 ms, speed-up 2.78×)
  ── t2 breakdown ──
    normal         : 0.002144 ms
    Add            : 0.001153 ms
    Sqrt           : 0.000928 ms
    Div            : 0.002336 ms
    Mul            : 0.002272 ms
    Add            : 0.002368 ms

[54] auto_fusion           │ /encoder/layer.4/intermediate/dense/MatMul
  t1 fused    : 0.034689 ms
  t2 separate : 0.040737 ms  (Δ 0.006048 ms, speed-up 1.17×)
  ── t2 breakdown ──
    normal         : 0.023841 ms
    Add            : 0.003552 ms
    Div            : 0.003808 ms
    Erf            : 0.002048 ms
    Add            : 0.002816 ms
    Mul            : 0.001856 ms
    Mul            : 0.002816 ms

[55] double_add            │ /encoder/layer.4/output/dense/MatMul
  t1 fused    : 0.083490 ms
  t2 separate : 0.086146 ms  (Δ 0.002656 ms, speed-up 1.03×)
  ── t2 breakdown ──
    normal         : 0.082562 ms
    Add            : 0.002336 ms
    Add            : 0.001248 ms

[56] sub_fusion            │ /encoder/layer.4/output/LayerNorm/ReduceMean
  t1 fused    : 0.003104 ms
  t2 separate : 0.004128 ms  (Δ 0.001024 ms, speed-up 1.33×)
  ── t2 breakdown ──
    normal         : 0.002176 ms
    Sub            : 0.001952 ms

[57] normalization_fusion  │ /encoder/layer.4/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.004001 ms
  t2 separate : 0.011231 ms  (Δ 0.007230 ms, speed-up 2.81×)
  ── t2 breakdown ──
    normal         : 0.002144 ms
    Add            : 0.001151 ms
    Sqrt           : 0.000896 ms
    Div            : 0.002304 ms
    Mul            : 0.002336 ms
    Add            : 0.002400 ms

[58] Add                   │ /encoder/layer.5/attention/self/query/MatMul
  t1 fused    : 0.023425 ms
  t2 separate : 0.025185 ms  (Δ 0.001760 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022881 ms
    Add            : 0.002304 ms

[59] Add                   │ /encoder/layer.5/attention/self/key/MatMul
  t1 fused    : 0.023520 ms
  t2 separate : 0.025216 ms  (Δ 0.001696 ms, speed-up 1.07×)
  ── t2 breakdown ──
    normal         : 0.022880 ms
    Add            : 0.002336 ms

[60] Add                   │ /encoder/layer.5/attention/self/value/MatMul
  t1 fused    : 0.023616 ms
  t2 separate : 0.025122 ms  (Δ 0.001506 ms, speed-up 1.06×)
  ── t2 breakdown ──
    normal         : 0.022849 ms
    Add            : 0.002273 ms

[61] div+add               │ /encoder/layer.5/attention/self/MatMul
  t1 fused    : 0.005568 ms
  t2 separate : 0.007040 ms  (Δ 0.001472 ms, speed-up 1.26×)
  ── t2 breakdown ──
    normal         : 0.004544 ms
    Div            : 0.002496 ms

[62] double_add            │ /encoder/layer.5/attention/output/dense/MatMul
  t1 fused    : 0.023648 ms
  t2 separate : 0.026370 ms  (Δ 0.002722 ms, speed-up 1.12×)
  ── t2 breakdown ──
    normal         : 0.022913 ms
    Add            : 0.002240 ms
    Add            : 0.001217 ms

[63] sub_fusion            │ /encoder/layer.5/attention/output/LayerNorm/ReduceMean
  t1 fused    : 0.003104 ms
  t2 separate : 0.004096 ms  (Δ 0.000992 ms, speed-up 1.32×)
  ── t2 breakdown ──
    normal         : 0.002144 ms
    Sub            : 0.001952 ms

[64] normalization_fusion  │ /encoder/layer.5/attention/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.004033 ms
  t2 separate : 0.011360 ms  (Δ 0.007327 ms, speed-up 2.82×)
  ── t2 breakdown ──
    normal         : 0.002112 ms
    Add            : 0.001152 ms
    Sqrt           : 0.000928 ms
    Div            : 0.002304 ms
    Mul            : 0.002368 ms
    Add            : 0.002496 ms

[65] auto_fusion           │ /encoder/layer.5/intermediate/dense/MatMul
  t1 fused    : 0.034657 ms
  t2 separate : 0.041344 ms  (Δ 0.006687 ms, speed-up 1.19×)
  ── t2 breakdown ──
    normal         : 0.024064 ms
    Add            : 0.003872 ms
    Div            : 0.003808 ms
    Erf            : 0.002080 ms
    Add            : 0.002848 ms
    Mul            : 0.001824 ms
    Mul            : 0.002848 ms

[66] double_add            │ /encoder/layer.5/output/dense/MatMul
  t1 fused    : 0.083490 ms
  t2 separate : 0.086178 ms  (Δ 0.002688 ms, speed-up 1.03×)
  ── t2 breakdown ──
    normal         : 0.082626 ms
    Add            : 0.002304 ms
    Add            : 0.001248 ms

[67] sub_fusion            │ /encoder/layer.5/output/LayerNorm/ReduceMean
  t1 fused    : 0.003104 ms
  t2 separate : 0.004160 ms  (Δ 0.001056 ms, speed-up 1.34×)
  ── t2 breakdown ──
    normal         : 0.002208 ms
    Sub            : 0.001952 ms

[68] normalization_fusion  │ /encoder/layer.5/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.004000 ms
  t2 separate : 0.011136 ms  (Δ 0.007136 ms, speed-up 2.78×)
  ── t2 breakdown ──
    normal         : 0.002112 ms
    Add            : 0.001152 ms
    Sqrt           : 0.000896 ms
    Div            : 0.002336 ms
    Mul            : 0.002272 ms
    Add            : 0.002368 ms

[69] Add                   │ /encoder/layer.6/attention/self/query/MatMul
  t1 fused    : 0.023425 ms
  t2 separate : 0.025186 ms  (Δ 0.001761 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022945 ms
    Add            : 0.002241 ms

[70] Add                   │ /encoder/layer.6/attention/self/key/MatMul
  t1 fused    : 0.023425 ms
  t2 separate : 0.025185 ms  (Δ 0.001760 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022881 ms
    Add            : 0.002304 ms

[71] Add                   │ /encoder/layer.6/attention/self/value/MatMul
  t1 fused    : 0.023425 ms
  t2 separate : 0.025121 ms  (Δ 0.001696 ms, speed-up 1.07×)
  ── t2 breakdown ──
    normal         : 0.022881 ms
    Add            : 0.002240 ms

[72] div+add               │ /encoder/layer.6/attention/self/MatMul
  t1 fused    : 0.005568 ms
  t2 separate : 0.007041 ms  (Δ 0.001473 ms, speed-up 1.26×)
  ── t2 breakdown ──
    normal         : 0.004576 ms
    Div            : 0.002465 ms

[73] double_add            │ /encoder/layer.6/attention/output/dense/MatMul
  t1 fused    : 0.023712 ms
  t2 separate : 0.026752 ms  (Δ 0.003040 ms, speed-up 1.13×)
  ── t2 breakdown ──
    normal         : 0.023232 ms
    Add            : 0.002304 ms
    Add            : 0.001216 ms

[74] sub_fusion            │ /encoder/layer.6/attention/output/LayerNorm/ReduceMean
  t1 fused    : 0.003072 ms
  t2 separate : 0.004096 ms  (Δ 0.001024 ms, speed-up 1.33×)
  ── t2 breakdown ──
    normal         : 0.002144 ms
    Sub            : 0.001952 ms

[75] normalization_fusion  │ /encoder/layer.6/attention/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.004032 ms
  t2 separate : 0.011137 ms  (Δ 0.007105 ms, speed-up 2.76×)
  ── t2 breakdown ──
    normal         : 0.002144 ms
    Add            : 0.001152 ms
    Sqrt           : 0.000896 ms
    Div            : 0.002336 ms
    Mul            : 0.002304 ms
    Add            : 0.002305 ms

[76] auto_fusion           │ /encoder/layer.6/intermediate/dense/MatMul
  t1 fused    : 0.034593 ms
  t2 separate : 0.041185 ms  (Δ 0.006592 ms, speed-up 1.19×)
  ── t2 breakdown ──
    normal         : 0.023873 ms
    Add            : 0.003872 ms
    Div            : 0.003840 ms
    Erf            : 0.002080 ms
    Add            : 0.002816 ms
    Mul            : 0.001888 ms
    Mul            : 0.002816 ms

[77] double_add            │ /encoder/layer.6/output/dense/MatMul
  t1 fused    : 0.083426 ms
  t2 separate : 0.086146 ms  (Δ 0.002720 ms, speed-up 1.03×)
  ── t2 breakdown ──
    normal         : 0.082626 ms
    Add            : 0.002304 ms
    Add            : 0.001216 ms

[78] sub_fusion            │ /encoder/layer.6/output/LayerNorm/ReduceMean
  t1 fused    : 0.003104 ms
  t2 separate : 0.004128 ms  (Δ 0.001024 ms, speed-up 1.33×)
  ── t2 breakdown ──
    normal         : 0.002176 ms
    Sub            : 0.001952 ms

[79] normalization_fusion  │ /encoder/layer.6/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.004000 ms
  t2 separate : 0.011073 ms  (Δ 0.007073 ms, speed-up 2.77×)
  ── t2 breakdown ──
    normal         : 0.002144 ms
    Add            : 0.001152 ms
    Sqrt           : 0.000896 ms
    Div            : 0.002337 ms
    Mul            : 0.002304 ms
    Add            : 0.002240 ms

[80] Add                   │ /encoder/layer.7/attention/self/query/MatMul
  t1 fused    : 0.023425 ms
  t2 separate : 0.025216 ms  (Δ 0.001791 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022880 ms
    Add            : 0.002336 ms

[81] Add                   │ /encoder/layer.7/attention/self/key/MatMul
  t1 fused    : 0.023585 ms
  t2 separate : 0.025089 ms  (Δ 0.001504 ms, speed-up 1.06×)
  ── t2 breakdown ──
    normal         : 0.022849 ms
    Add            : 0.002240 ms

[82] Add                   │ /encoder/layer.7/attention/self/value/MatMul
  t1 fused    : 0.023361 ms
  t2 separate : 0.025153 ms  (Δ 0.001792 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022881 ms
    Add            : 0.002272 ms

[83] div+add               │ /encoder/layer.7/attention/self/MatMul
  t1 fused    : 0.005600 ms
  t2 separate : 0.007041 ms  (Δ 0.001441 ms, speed-up 1.26×)
  ── t2 breakdown ──
    normal         : 0.004576 ms
    Div            : 0.002465 ms

[84] double_add            │ /encoder/layer.7/attention/output/dense/MatMul
  t1 fused    : 0.023777 ms
  t2 separate : 0.026529 ms  (Δ 0.002752 ms, speed-up 1.12×)
  ── t2 breakdown ──
    normal         : 0.023041 ms
    Add            : 0.002240 ms
    Add            : 0.001248 ms

[85] sub_fusion            │ /encoder/layer.7/attention/output/LayerNorm/ReduceMean
  t1 fused    : 0.003072 ms
  t2 separate : 0.004064 ms  (Δ 0.000992 ms, speed-up 1.32×)
  ── t2 breakdown ──
    normal         : 0.002112 ms
    Sub            : 0.001952 ms

[86] normalization_fusion  │ /encoder/layer.7/attention/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.004032 ms
  t2 separate : 0.011136 ms  (Δ 0.007104 ms, speed-up 2.76×)
  ── t2 breakdown ──
    normal         : 0.002144 ms
    Add            : 0.001152 ms
    Sqrt           : 0.000928 ms
    Div            : 0.002304 ms
    Mul            : 0.002304 ms
    Add            : 0.002304 ms

[87] auto_fusion           │ /encoder/layer.7/intermediate/dense/MatMul
  t1 fused    : 0.034817 ms
  t2 separate : 0.041857 ms  (Δ 0.007040 ms, speed-up 1.20×)
  ── t2 breakdown ──
    normal         : 0.024512 ms
    Add            : 0.003968 ms
    Div            : 0.003809 ms
    Erf            : 0.002048 ms
    Add            : 0.002816 ms
    Mul            : 0.001856 ms
    Mul            : 0.002848 ms

[88] double_add            │ /encoder/layer.7/output/dense/MatMul
  t1 fused    : 0.083426 ms
  t2 separate : 0.085986 ms  (Δ 0.002560 ms, speed-up 1.03×)
  ── t2 breakdown ──
    normal         : 0.082594 ms
    Add            : 0.002208 ms
    Add            : 0.001184 ms

[89] sub_fusion            │ /encoder/layer.7/output/LayerNorm/ReduceMean
  t1 fused    : 0.003072 ms
  t2 separate : 0.004192 ms  (Δ 0.001120 ms, speed-up 1.36×)
  ── t2 breakdown ──
    normal         : 0.002240 ms
    Sub            : 0.001952 ms

[90] normalization_fusion  │ /encoder/layer.7/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.004000 ms
  t2 separate : 0.011104 ms  (Δ 0.007104 ms, speed-up 2.78×)
  ── t2 breakdown ──
    normal         : 0.002112 ms
    Add            : 0.001152 ms
    Sqrt           : 0.000896 ms
    Div            : 0.002336 ms
    Mul            : 0.002304 ms
    Add            : 0.002304 ms

[91] Add                   │ /encoder/layer.8/attention/self/query/MatMul
  t1 fused    : 0.023521 ms
  t2 separate : 0.025216 ms  (Δ 0.001695 ms, speed-up 1.07×)
  ── t2 breakdown ──
    normal         : 0.022848 ms
    Add            : 0.002368 ms

[92] Add                   │ /encoder/layer.8/attention/self/key/MatMul
  t1 fused    : 0.023521 ms
  t2 separate : 0.025185 ms  (Δ 0.001664 ms, speed-up 1.07×)
  ── t2 breakdown ──
    normal         : 0.022881 ms
    Add            : 0.002304 ms

[93] Add                   │ /encoder/layer.8/attention/self/value/MatMul
  t1 fused    : 0.023393 ms
  t2 separate : 0.025345 ms  (Δ 0.001952 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022912 ms
    Add            : 0.002433 ms

[94] div+add               │ /encoder/layer.8/attention/self/MatMul
  t1 fused    : 0.005569 ms
  t2 separate : 0.007104 ms  (Δ 0.001535 ms, speed-up 1.28×)
  ── t2 breakdown ──
    normal         : 0.004608 ms
    Div            : 0.002496 ms

[95] double_add            │ /encoder/layer.8/attention/output/dense/MatMul
  t1 fused    : 0.023712 ms
  t2 separate : 0.026593 ms  (Δ 0.002881 ms, speed-up 1.12×)
  ── t2 breakdown ──
    normal         : 0.023073 ms
    Add            : 0.002336 ms
    Add            : 0.001184 ms

[96] sub_fusion            │ /encoder/layer.8/attention/output/LayerNorm/ReduceMean
  t1 fused    : 0.003104 ms
  t2 separate : 0.004095 ms  (Δ 0.000991 ms, speed-up 1.32×)
  ── t2 breakdown ──
    normal         : 0.002144 ms
    Sub            : 0.001951 ms

[97] normalization_fusion  │ /encoder/layer.8/attention/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.004032 ms
  t2 separate : 0.011298 ms  (Δ 0.007266 ms, speed-up 2.80×)
  ── t2 breakdown ──
    normal         : 0.002144 ms
    Add            : 0.001152 ms
    Sqrt           : 0.000929 ms
    Div            : 0.002304 ms
    Mul            : 0.002432 ms
    Add            : 0.002337 ms

[98] auto_fusion           │ /encoder/layer.8/intermediate/dense/MatMul
  t1 fused    : 0.034753 ms
  t2 separate : 0.041761 ms  (Δ 0.007008 ms, speed-up 1.20×)
  ── t2 breakdown ──
    normal         : 0.024513 ms
    Add            : 0.003872 ms
    Div            : 0.003840 ms
    Erf            : 0.002048 ms
    Add            : 0.002848 ms
    Mul            : 0.001792 ms
    Mul            : 0.002848 ms

[99] double_add            │ /encoder/layer.8/output/dense/MatMul
  t1 fused    : 0.083426 ms
  t2 separate : 0.086274 ms  (Δ 0.002848 ms, speed-up 1.03×)
  ── t2 breakdown ──
    normal         : 0.082690 ms
    Add            : 0.002368 ms
    Add            : 0.001216 ms

[100] sub_fusion            │ /encoder/layer.8/output/LayerNorm/ReduceMean
  t1 fused    : 0.003072 ms
  t2 separate : 0.004128 ms  (Δ 0.001056 ms, speed-up 1.34×)
  ── t2 breakdown ──
    normal         : 0.002208 ms
    Sub            : 0.001920 ms

[101] normalization_fusion  │ /encoder/layer.8/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.004033 ms
  t2 separate : 0.011200 ms  (Δ 0.007167 ms, speed-up 2.78×)
  ── t2 breakdown ──
    normal         : 0.002176 ms
    Add            : 0.001152 ms
    Sqrt           : 0.000896 ms
    Div            : 0.002336 ms
    Mul            : 0.002368 ms
    Add            : 0.002272 ms

[102] Add                   │ /encoder/layer.9/attention/self/query/MatMul
  t1 fused    : 0.023424 ms
  t2 separate : 0.025281 ms  (Δ 0.001857 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022977 ms
    Add            : 0.002304 ms

[103] Add                   │ /encoder/layer.9/attention/self/key/MatMul
  t1 fused    : 0.023392 ms
  t2 separate : 0.025184 ms  (Δ 0.001792 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022880 ms
    Add            : 0.002304 ms

[104] Add                   │ /encoder/layer.9/attention/self/value/MatMul
  t1 fused    : 0.023393 ms
  t2 separate : 0.025185 ms  (Δ 0.001792 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022945 ms
    Add            : 0.002240 ms

[105] div+add               │ /encoder/layer.9/attention/self/MatMul
  t1 fused    : 0.005568 ms
  t2 separate : 0.007073 ms  (Δ 0.001505 ms, speed-up 1.27×)
  ── t2 breakdown ──
    normal         : 0.004577 ms
    Div            : 0.002496 ms

[106] double_add            │ /encoder/layer.9/attention/output/dense/MatMul
  t1 fused    : 0.023713 ms
  t2 separate : 0.026400 ms  (Δ 0.002687 ms, speed-up 1.11×)
  ── t2 breakdown ──
    normal         : 0.022880 ms
    Add            : 0.002304 ms
    Add            : 0.001216 ms

[107] sub_fusion            │ /encoder/layer.9/attention/output/LayerNorm/ReduceMean
  t1 fused    : 0.003072 ms
  t2 separate : 0.004065 ms  (Δ 0.000993 ms, speed-up 1.32×)
  ── t2 breakdown ──
    normal         : 0.002113 ms
    Sub            : 0.001952 ms

[108] normalization_fusion  │ /encoder/layer.9/attention/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.004000 ms
  t2 separate : 0.011201 ms  (Δ 0.007201 ms, speed-up 2.80×)
  ── t2 breakdown ──
    normal         : 0.002112 ms
    Add            : 0.001153 ms
    Sqrt           : 0.000928 ms
    Div            : 0.002304 ms
    Mul            : 0.002336 ms
    Add            : 0.002368 ms

[109] auto_fusion           │ /encoder/layer.9/intermediate/dense/MatMul
  t1 fused    : 0.034849 ms
  t2 separate : 0.041185 ms  (Δ 0.006336 ms, speed-up 1.18×)
  ── t2 breakdown ──
    normal         : 0.023873 ms
    Add            : 0.003872 ms
    Div            : 0.003872 ms
    Erf            : 0.002048 ms
    Add            : 0.002848 ms
    Mul            : 0.001824 ms
    Mul            : 0.002848 ms

[110] double_add            │ /encoder/layer.9/output/dense/MatMul
  t1 fused    : 0.083426 ms
  t2 separate : 0.086081 ms  (Δ 0.002655 ms, speed-up 1.03×)
  ── t2 breakdown ──
    normal         : 0.082626 ms
    Add            : 0.002271 ms
    Add            : 0.001184 ms

[111] sub_fusion            │ /encoder/layer.9/output/LayerNorm/ReduceMean
  t1 fused    : 0.003072 ms
  t2 separate : 0.004160 ms  (Δ 0.001088 ms, speed-up 1.35×)
  ── t2 breakdown ──
    normal         : 0.002208 ms
    Sub            : 0.001952 ms

[112] normalization_fusion  │ /encoder/layer.9/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.004032 ms
  t2 separate : 0.011040 ms  (Δ 0.007008 ms, speed-up 2.74×)
  ── t2 breakdown ──
    normal         : 0.002112 ms
    Add            : 0.001152 ms
    Sqrt           : 0.000896 ms
    Div            : 0.002336 ms
    Mul            : 0.002240 ms
    Add            : 0.002304 ms

[113] Add                   │ /encoder/layer.10/attention/self/query/MatMul
  t1 fused    : 0.023392 ms
  t2 separate : 0.025184 ms  (Δ 0.001792 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022880 ms
    Add            : 0.002304 ms

[114] Add                   │ /encoder/layer.10/attention/self/key/MatMul
  t1 fused    : 0.023424 ms
  t2 separate : 0.025281 ms  (Δ 0.001857 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022881 ms
    Add            : 0.002400 ms

[115] Add                   │ /encoder/layer.10/attention/self/value/MatMul
  t1 fused    : 0.023681 ms
  t2 separate : 0.025281 ms  (Δ 0.001600 ms, speed-up 1.07×)
  ── t2 breakdown ──
    normal         : 0.022913 ms
    Add            : 0.002368 ms

[116] div+add               │ /encoder/layer.10/attention/self/MatMul
  t1 fused    : 0.005568 ms
  t2 separate : 0.007008 ms  (Δ 0.001440 ms, speed-up 1.26×)
  ── t2 breakdown ──
    normal         : 0.004544 ms
    Div            : 0.002464 ms

[117] double_add            │ /encoder/layer.10/attention/output/dense/MatMul
  t1 fused    : 0.023649 ms
  t2 separate : 0.026336 ms  (Δ 0.002687 ms, speed-up 1.11×)
  ── t2 breakdown ──
    normal         : 0.022849 ms
    Add            : 0.002272 ms
    Add            : 0.001215 ms

[118] sub_fusion            │ /encoder/layer.10/attention/output/LayerNorm/ReduceMean
  t1 fused    : 0.003104 ms
  t2 separate : 0.004096 ms  (Δ 0.000992 ms, speed-up 1.32×)
  ── t2 breakdown ──
    normal         : 0.002144 ms
    Sub            : 0.001952 ms

[119] normalization_fusion  │ /encoder/layer.10/attention/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.004032 ms
  t2 separate : 0.011168 ms  (Δ 0.007136 ms, speed-up 2.77×)
  ── t2 breakdown ──
    normal         : 0.002144 ms
    Add            : 0.001184 ms
    Sqrt           : 0.000928 ms
    Div            : 0.002304 ms
    Mul            : 0.002304 ms
    Add            : 0.002304 ms

[120] auto_fusion           │ /encoder/layer.10/intermediate/dense/MatMul
  t1 fused    : 0.034753 ms
  t2 separate : 0.041345 ms  (Δ 0.006592 ms, speed-up 1.19×)
  ── t2 breakdown ──
    normal         : 0.024193 ms
    Add            : 0.003776 ms
    Div            : 0.003808 ms
    Erf            : 0.002048 ms
    Add            : 0.002816 ms
    Mul            : 0.001888 ms
    Mul            : 0.002816 ms

[121] double_add            │ /encoder/layer.10/output/dense/MatMul
  t1 fused    : 0.083490 ms
  t2 separate : 0.086147 ms  (Δ 0.002657 ms, speed-up 1.03×)
  ── t2 breakdown ──
    normal         : 0.082626 ms
    Add            : 0.002305 ms
    Add            : 0.001216 ms

[122] sub_fusion            │ /encoder/layer.10/output/LayerNorm/ReduceMean
  t1 fused    : 0.003104 ms
  t2 separate : 0.004128 ms  (Δ 0.001024 ms, speed-up 1.33×)
  ── t2 breakdown ──
    normal         : 0.002208 ms
    Sub            : 0.001920 ms

[123] normalization_fusion  │ /encoder/layer.10/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.004000 ms
  t2 separate : 0.011104 ms  (Δ 0.007104 ms, speed-up 2.78×)
  ── t2 breakdown ──
    normal         : 0.002144 ms
    Add            : 0.001152 ms
    Sqrt           : 0.000896 ms
    Div            : 0.002304 ms
    Mul            : 0.002304 ms
    Add            : 0.002304 ms

[124] Add                   │ /encoder/layer.11/attention/self/query/MatMul
  t1 fused    : 0.023425 ms
  t2 separate : 0.025185 ms  (Δ 0.001760 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022849 ms
    Add            : 0.002336 ms

[125] Add                   │ /encoder/layer.11/attention/self/key/MatMul
  t1 fused    : 0.023392 ms
  t2 separate : 0.025249 ms  (Δ 0.001857 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022881 ms
    Add            : 0.002368 ms

[126] Add                   │ /encoder/layer.11/attention/self/value/MatMul
  t1 fused    : 0.023392 ms
  t2 separate : 0.025152 ms  (Δ 0.001760 ms, speed-up 1.08×)
  ── t2 breakdown ──
    normal         : 0.022881 ms
    Add            : 0.002271 ms

[127] div+add               │ /encoder/layer.11/attention/self/MatMul
  t1 fused    : 0.005568 ms
  t2 separate : 0.007008 ms  (Δ 0.001440 ms, speed-up 1.26×)
  ── t2 breakdown ──
    normal         : 0.004544 ms
    Div            : 0.002464 ms

[128] double_add            │ /encoder/layer.11/attention/output/dense/MatMul
  t1 fused    : 0.023777 ms
  t2 separate : 0.026432 ms  (Δ 0.002655 ms, speed-up 1.11×)
  ── t2 breakdown ──
    normal         : 0.022912 ms
    Add            : 0.002304 ms
    Add            : 0.001216 ms

[129] sub_fusion            │ /encoder/layer.11/attention/output/LayerNorm/ReduceMean
  t1 fused    : 0.003072 ms
  t2 separate : 0.004032 ms  (Δ 0.000960 ms, speed-up 1.31×)
  ── t2 breakdown ──
    normal         : 0.002112 ms
    Sub            : 0.001920 ms

[130] normalization_fusion  │ /encoder/layer.11/attention/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.004000 ms
  t2 separate : 0.011200 ms  (Δ 0.007200 ms, speed-up 2.80×)
  ── t2 breakdown ──
    normal         : 0.002176 ms
    Add            : 0.001152 ms
    Sqrt           : 0.000928 ms
    Div            : 0.002336 ms
    Mul            : 0.002208 ms
    Add            : 0.002400 ms

[131] auto_fusion           │ /encoder/layer.11/intermediate/dense/MatMul
  t1 fused    : 0.034817 ms
  t2 separate : 0.041088 ms  (Δ 0.006271 ms, speed-up 1.18×)
  ── t2 breakdown ──
    normal         : 0.023904 ms
    Add            : 0.003808 ms
    Div            : 0.003840 ms
    Erf            : 0.002016 ms
    Add            : 0.002816 ms
    Mul            : 0.001856 ms
    Mul            : 0.002848 ms

[132] double_add            │ /encoder/layer.11/output/dense/MatMul
  t1 fused    : 0.083427 ms
  t2 separate : 0.086114 ms  (Δ 0.002687 ms, speed-up 1.03×)
  ── t2 breakdown ──
    normal         : 0.082594 ms
    Add            : 0.002304 ms
    Add            : 0.001216 ms

[133] sub_fusion            │ /encoder/layer.11/output/LayerNorm/ReduceMean
  t1 fused    : 0.003072 ms
  t2 separate : 0.004160 ms  (Δ 0.001088 ms, speed-up 1.35×)
  ── t2 breakdown ──
    normal         : 0.002208 ms
    Sub            : 0.001952 ms

[134] normalization_fusion  │ /encoder/layer.11/output/LayerNorm/ReduceMean_1
  t1 fused    : 0.004000 ms
  t2 separate : 0.011202 ms  (Δ 0.007202 ms, speed-up 2.80×)
  ── t2 breakdown ──
    normal         : 0.002145 ms
    Add            : 0.001152 ms
    Sqrt           : 0.000896 ms
    Div            : 0.002304 ms
    Mul            : 0.002337 ms
    Add            : 0.002368 ms

===================================================================================================================
► 每個『組合類型』總和
  sub_fusion            : fused 0.077346 ms   sep 0.102752 ms   Δ 0.025406 ms   speed-up 1.33×
  normalization_fusion  : fused 0.100419 ms   sep 0.278862 ms   Δ 0.178443 ms   speed-up 2.78×
  Add                   : fused 0.843893 ms   sep 0.907709 ms   Δ 0.063816 ms   speed-up 1.08×
  div+add               : fused 0.066882 ms   sep 0.084548 ms   Δ 0.017666 ms   speed-up 1.26×
  double_add            : fused 1.285888 ms   sep 1.351396 ms   Δ 0.065508 ms   speed-up 1.05×
  auto_fusion           : fused 0.416717 ms   sep 0.494731 ms   Δ 0.078014 ms   speed-up 1.19×

===================================================================================================================
► 四組合累計 Σ
  Σ fused     : 2.791145 ms
  Σ separate  : 3.219998 ms
  Δ (sep-fuse): 0.428853 ms   speed-up 1.15×

===================================================================================================================
► 其餘 Run_with_type 彙總 (未納入組合)
  Add            : t1     0.004736 ms │ t2     0.031775 ms │ Δ     0.027039 ms
  Cast           : t1     0.001248 ms │ t2     0.001280 ms │ Δ     0.000032 ms
  CumSum         : t1     0.021569 ms │ t2     0.021537 ms │ Δ    -0.000032 ms
  Equal          : t1     0.001473 ms │ t2     0.001440 ms │ Δ    -0.000033 ms
  Gather         : t1     0.076227 ms │ t2     0.082146 ms │ Δ     0.005919 ms
  Gemm           : t1     0.035297 ms │ t2     0.032833 ms │ Δ    -0.002464 ms
  Mul            : t1     0.002625 ms │ t2     0.002751 ms │ Δ     0.000126 ms
  Not            : t1     0.000960 ms │ t2     0.000960 ms │ Δ     0.000000 ms
  Pow            : t1     0.035649 ms │ t2     0.035521 ms │ Δ    -0.000128 ms
  Softmax        : t1     0.031200 ms │ t2     0.031268 ms │ Δ     0.000068 ms
  Sub            : t1     0.001184 ms │ t2     0.001152 ms │ Δ    -0.000032 ms
  Tanh           : t1     0.000960 ms │ t2     0.000961 ms │ Δ     0.000001 ms
  Transpose      : t1     0.099461 ms │ t2     0.100450 ms │ Δ     0.000989 ms

===================================================================================================================
► 子-op (含 normal) 跨全部組合累積時間
  normal         : 2.541801 ms
  Add            : 0.334635 ms
  Div            : 0.133669 ms
  Mul            : 0.113892 ms
  Sub            : 0.048640 ms
  Erf            : 0.024608 ms
  Sqrt           : 0.022753 ms

===================================================================================================================
► 整體檔案總執行時間
  t1 total : 3.177336 ms
  t2 total : 3.637417 ms
  Δ        : 0.460081 ms   speed-up 1.14×
Editor is loading...
Leave a Comment