Untitled
unknown
batchfile
a year ago
47 kB
17
Indexable
===================================================================================================================
► 每個『組合實例』(t1 fused vs. t2 separate)
[1] sub_fusion │ /embeddings/LayerNorm/ReduceMean
t1 fused : 0.003200 ms
t2 separate : 0.004256 ms (Δ 0.001056 ms, speed-up 1.33×)
── t2 breakdown ──
normal : 0.002208 ms
Sub : 0.002048 ms
[2] normalization_fusion │ /embeddings/LayerNorm/ReduceMean_1
t1 fused : 0.004065 ms
t2 separate : 0.011074 ms (Δ 0.007009 ms, speed-up 2.72×)
── t2 breakdown ──
normal : 0.002016 ms
Add : 0.001153 ms
Sqrt : 0.000928 ms
Div : 0.002368 ms
Mul : 0.002241 ms
Add : 0.002368 ms
[3] Add │ /encoder/layer.0/attention/self/query/MatMul
t1 fused : 0.023361 ms
t2 separate : 0.025185 ms (Δ 0.001824 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022881 ms
Add : 0.002304 ms
[4] Add │ /encoder/layer.0/attention/self/key/MatMul
t1 fused : 0.023361 ms
t2 separate : 0.025281 ms (Δ 0.001920 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022913 ms
Add : 0.002368 ms
[5] Add │ /encoder/layer.0/attention/self/value/MatMul
t1 fused : 0.023425 ms
t2 separate : 0.025185 ms (Δ 0.001760 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022849 ms
Add : 0.002336 ms
[6] div+add │ /encoder/layer.0/attention/self/MatMul
t1 fused : 0.005568 ms
t2 separate : 0.007104 ms (Δ 0.001536 ms, speed-up 1.28×)
── t2 breakdown ──
normal : 0.004640 ms
Div : 0.002464 ms
[7] double_add │ /encoder/layer.0/attention/output/dense/MatMul
t1 fused : 0.023712 ms
t2 separate : 0.026402 ms (Δ 0.002690 ms, speed-up 1.11×)
── t2 breakdown ──
normal : 0.022881 ms
Add : 0.002304 ms
Add : 0.001217 ms
[8] sub_fusion │ /encoder/layer.0/attention/output/LayerNorm/ReduceMean
t1 fused : 0.003104 ms
t2 separate : 0.004032 ms (Δ 0.000928 ms, speed-up 1.30×)
── t2 breakdown ──
normal : 0.002112 ms
Sub : 0.001920 ms
[9] normalization_fusion │ /encoder/layer.0/attention/output/LayerNorm/ReduceMean_1
t1 fused : 0.004032 ms
t2 separate : 0.011168 ms (Δ 0.007136 ms, speed-up 2.77×)
── t2 breakdown ──
normal : 0.002048 ms
Add : 0.001184 ms
Sqrt : 0.000928 ms
Div : 0.002304 ms
Mul : 0.002368 ms
Add : 0.002336 ms
[10] auto_fusion │ /encoder/layer.0/intermediate/dense/MatMul
t1 fused : 0.034786 ms
t2 separate : 0.040769 ms (Δ 0.005983 ms, speed-up 1.17×)
── t2 breakdown ──
normal : 0.023649 ms
Add : 0.003776 ms
Div : 0.003808 ms
Erf : 0.002048 ms
Add : 0.002816 ms
Mul : 0.001824 ms
Mul : 0.002848 ms
[11] double_add │ /encoder/layer.0/output/dense/MatMul
t1 fused : 0.083362 ms
t2 separate : 0.086146 ms (Δ 0.002784 ms, speed-up 1.03×)
── t2 breakdown ──
normal : 0.082722 ms
Add : 0.002208 ms
Add : 0.001216 ms
[12] sub_fusion │ /encoder/layer.0/output/LayerNorm/ReduceMean
t1 fused : 0.003104 ms
t2 separate : 0.004064 ms (Δ 0.000960 ms, speed-up 1.31×)
── t2 breakdown ──
normal : 0.002112 ms
Sub : 0.001952 ms
[13] normalization_fusion │ /encoder/layer.0/output/LayerNorm/ReduceMean_1
t1 fused : 0.004032 ms
t2 separate : 0.011104 ms (Δ 0.007072 ms, speed-up 2.75×)
── t2 breakdown ──
normal : 0.002144 ms
Add : 0.001152 ms
Sqrt : 0.000896 ms
Div : 0.002336 ms
Mul : 0.002240 ms
Add : 0.002336 ms
[14] Add │ /encoder/layer.1/attention/self/query/MatMul
t1 fused : 0.023329 ms
t2 separate : 0.025153 ms (Δ 0.001824 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022881 ms
Add : 0.002272 ms
[15] Add │ /encoder/layer.1/attention/self/key/MatMul
t1 fused : 0.023424 ms
t2 separate : 0.025249 ms (Δ 0.001825 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022913 ms
Add : 0.002336 ms
[16] Add │ /encoder/layer.1/attention/self/value/MatMul
t1 fused : 0.023361 ms
t2 separate : 0.025153 ms (Δ 0.001792 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022880 ms
Add : 0.002273 ms
[17] div+add │ /encoder/layer.1/attention/self/MatMul
t1 fused : 0.005601 ms
t2 separate : 0.007008 ms (Δ 0.001407 ms, speed-up 1.25×)
── t2 breakdown ──
normal : 0.004544 ms
Div : 0.002464 ms
[18] double_add │ /encoder/layer.1/attention/output/dense/MatMul
t1 fused : 0.023649 ms
t2 separate : 0.026402 ms (Δ 0.002753 ms, speed-up 1.12×)
── t2 breakdown ──
normal : 0.022881 ms
Add : 0.002305 ms
Add : 0.001216 ms
[19] sub_fusion │ /encoder/layer.1/attention/output/LayerNorm/ReduceMean
t1 fused : 0.003073 ms
t2 separate : 0.004097 ms (Δ 0.001024 ms, speed-up 1.33×)
── t2 breakdown ──
normal : 0.002144 ms
Sub : 0.001953 ms
[20] normalization_fusion │ /encoder/layer.1/attention/output/LayerNorm/ReduceMean_1
t1 fused : 0.004000 ms
t2 separate : 0.011008 ms (Δ 0.007008 ms, speed-up 2.75×)
── t2 breakdown ──
normal : 0.002048 ms
Add : 0.001152 ms
Sqrt : 0.000896 ms
Div : 0.002336 ms
Mul : 0.002304 ms
Add : 0.002272 ms
[21] auto_fusion │ /encoder/layer.1/intermediate/dense/MatMul
t1 fused : 0.034625 ms
t2 separate : 0.040994 ms (Δ 0.006369 ms, speed-up 1.18×)
── t2 breakdown ──
normal : 0.023937 ms
Add : 0.003680 ms
Div : 0.003841 ms
Erf : 0.002048 ms
Add : 0.002816 ms
Mul : 0.001856 ms
Mul : 0.002816 ms
[22] double_add │ /encoder/layer.1/output/dense/MatMul
t1 fused : 0.083554 ms
t2 separate : 0.086306 ms (Δ 0.002752 ms, speed-up 1.03×)
── t2 breakdown ──
normal : 0.082818 ms
Add : 0.002272 ms
Add : 0.001216 ms
[23] sub_fusion │ /encoder/layer.1/output/LayerNorm/ReduceMean
t1 fused : 0.003104 ms
t2 separate : 0.004096 ms (Δ 0.000992 ms, speed-up 1.32×)
── t2 breakdown ──
normal : 0.002176 ms
Sub : 0.001920 ms
[24] normalization_fusion │ /encoder/layer.1/output/LayerNorm/ReduceMean_1
t1 fused : 0.004000 ms
t2 separate : 0.011041 ms (Δ 0.007041 ms, speed-up 2.76×)
── t2 breakdown ──
normal : 0.002112 ms
Add : 0.001152 ms
Sqrt : 0.000896 ms
Div : 0.002304 ms
Mul : 0.002304 ms
Add : 0.002273 ms
[25] Add │ /encoder/layer.2/attention/self/query/MatMul
t1 fused : 0.023552 ms
t2 separate : 0.025664 ms (Δ 0.002112 ms, speed-up 1.09×)
── t2 breakdown ──
normal : 0.023296 ms
Add : 0.002368 ms
[26] Add │ /encoder/layer.2/attention/self/key/MatMul
t1 fused : 0.023424 ms
t2 separate : 0.025313 ms (Δ 0.001889 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022881 ms
Add : 0.002432 ms
[27] Add │ /encoder/layer.2/attention/self/value/MatMul
t1 fused : 0.023393 ms
t2 separate : 0.025153 ms (Δ 0.001760 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022849 ms
Add : 0.002304 ms
[28] div+add │ /encoder/layer.2/attention/self/MatMul
t1 fused : 0.005568 ms
t2 separate : 0.007041 ms (Δ 0.001473 ms, speed-up 1.26×)
── t2 breakdown ──
normal : 0.004545 ms
Div : 0.002496 ms
[29] double_add │ /encoder/layer.2/attention/output/dense/MatMul
t1 fused : 0.023649 ms
t2 separate : 0.026465 ms (Δ 0.002816 ms, speed-up 1.12×)
── t2 breakdown ──
normal : 0.022913 ms
Add : 0.002304 ms
Add : 0.001248 ms
[30] sub_fusion │ /encoder/layer.2/attention/output/LayerNorm/ReduceMean
t1 fused : 0.003072 ms
t2 separate : 0.004031 ms (Δ 0.000959 ms, speed-up 1.31×)
── t2 breakdown ──
normal : 0.002111 ms
Sub : 0.001920 ms
[31] normalization_fusion │ /encoder/layer.2/attention/output/LayerNorm/ReduceMean_1
t1 fused : 0.004000 ms
t2 separate : 0.011168 ms (Δ 0.007168 ms, speed-up 2.79×)
── t2 breakdown ──
normal : 0.002144 ms
Add : 0.001152 ms
Sqrt : 0.000928 ms
Div : 0.002336 ms
Mul : 0.002304 ms
Add : 0.002304 ms
[32] auto_fusion │ /encoder/layer.2/intermediate/dense/MatMul
t1 fused : 0.034753 ms
t2 separate : 0.041345 ms (Δ 0.006592 ms, speed-up 1.19×)
── t2 breakdown ──
normal : 0.024033 ms
Add : 0.003936 ms
Div : 0.003808 ms
Erf : 0.002048 ms
Add : 0.002848 ms
Mul : 0.001856 ms
Mul : 0.002816 ms
[33] double_add │ /encoder/layer.2/output/dense/MatMul
t1 fused : 0.083426 ms
t2 separate : 0.086211 ms (Δ 0.002785 ms, speed-up 1.03×)
── t2 breakdown ──
normal : 0.082595 ms
Add : 0.002400 ms
Add : 0.001216 ms
[34] sub_fusion │ /encoder/layer.2/output/LayerNorm/ReduceMean
t1 fused : 0.003105 ms
t2 separate : 0.004128 ms (Δ 0.001023 ms, speed-up 1.33×)
── t2 breakdown ──
normal : 0.002208 ms
Sub : 0.001920 ms
[35] normalization_fusion │ /encoder/layer.2/output/LayerNorm/ReduceMean_1
t1 fused : 0.003999 ms
t2 separate : 0.011265 ms (Δ 0.007266 ms, speed-up 2.82×)
── t2 breakdown ──
normal : 0.002145 ms
Add : 0.001152 ms
Sqrt : 0.000896 ms
Div : 0.002336 ms
Mul : 0.002368 ms
Add : 0.002368 ms
[36] Add │ /encoder/layer.3/attention/self/query/MatMul
t1 fused : 0.023393 ms
t2 separate : 0.025153 ms (Δ 0.001760 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022881 ms
Add : 0.002272 ms
[37] Add │ /encoder/layer.3/attention/self/key/MatMul
t1 fused : 0.023488 ms
t2 separate : 0.025281 ms (Δ 0.001793 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022913 ms
Add : 0.002368 ms
[38] Add │ /encoder/layer.3/attention/self/value/MatMul
t1 fused : 0.023520 ms
t2 separate : 0.025281 ms (Δ 0.001761 ms, speed-up 1.07×)
── t2 breakdown ──
normal : 0.022880 ms
Add : 0.002401 ms
[39] div+add │ /encoder/layer.3/attention/self/MatMul
t1 fused : 0.005568 ms
t2 separate : 0.007040 ms (Δ 0.001472 ms, speed-up 1.26×)
── t2 breakdown ──
normal : 0.004608 ms
Div : 0.002432 ms
[40] double_add │ /encoder/layer.3/attention/output/dense/MatMul
t1 fused : 0.023712 ms
t2 separate : 0.026369 ms (Δ 0.002657 ms, speed-up 1.11×)
── t2 breakdown ──
normal : 0.022881 ms
Add : 0.002304 ms
Add : 0.001184 ms
[41] sub_fusion │ /encoder/layer.3/attention/output/LayerNorm/ReduceMean
t1 fused : 0.003072 ms
t2 separate : 0.004096 ms (Δ 0.001024 ms, speed-up 1.33×)
── t2 breakdown ──
normal : 0.002144 ms
Sub : 0.001952 ms
[42] normalization_fusion │ /encoder/layer.3/attention/output/LayerNorm/ReduceMean_1
t1 fused : 0.004000 ms
t2 separate : 0.011170 ms (Δ 0.007170 ms, speed-up 2.79×)
── t2 breakdown ──
normal : 0.002113 ms
Add : 0.001152 ms
Sqrt : 0.000928 ms
Div : 0.002304 ms
Mul : 0.002305 ms
Add : 0.002368 ms
[43] auto_fusion │ /encoder/layer.3/intermediate/dense/MatMul
t1 fused : 0.034625 ms
t2 separate : 0.041121 ms (Δ 0.006496 ms, speed-up 1.19×)
── t2 breakdown ──
normal : 0.023969 ms
Add : 0.003776 ms
Div : 0.003840 ms
Erf : 0.002048 ms
Add : 0.002816 ms
Mul : 0.001824 ms
Mul : 0.002848 ms
[44] double_add │ /encoder/layer.3/output/dense/MatMul
t1 fused : 0.083394 ms
t2 separate : 0.086178 ms (Δ 0.002784 ms, speed-up 1.03×)
── t2 breakdown ──
normal : 0.082690 ms
Add : 0.002272 ms
Add : 0.001216 ms
[45] sub_fusion │ /encoder/layer.3/output/LayerNorm/ReduceMean
t1 fused : 0.003104 ms
t2 separate : 0.004128 ms (Δ 0.001024 ms, speed-up 1.33×)
── t2 breakdown ──
normal : 0.002208 ms
Sub : 0.001920 ms
[46] normalization_fusion │ /encoder/layer.3/output/LayerNorm/ReduceMean_1
t1 fused : 0.004032 ms
t2 separate : 0.011073 ms (Δ 0.007041 ms, speed-up 2.75×)
── t2 breakdown ──
normal : 0.002112 ms
Add : 0.001152 ms
Sqrt : 0.000896 ms
Div : 0.002304 ms
Mul : 0.002305 ms
Add : 0.002304 ms
[47] Add │ /encoder/layer.4/attention/self/query/MatMul
t1 fused : 0.023392 ms
t2 separate : 0.025153 ms (Δ 0.001761 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022849 ms
Add : 0.002304 ms
[48] Add │ /encoder/layer.4/attention/self/key/MatMul
t1 fused : 0.023456 ms
t2 separate : 0.025152 ms (Δ 0.001696 ms, speed-up 1.07×)
── t2 breakdown ──
normal : 0.022848 ms
Add : 0.002304 ms
[49] Add │ /encoder/layer.4/attention/self/value/MatMul
t1 fused : 0.023457 ms
t2 separate : 0.025152 ms (Δ 0.001695 ms, speed-up 1.07×)
── t2 breakdown ──
normal : 0.022912 ms
Add : 0.002240 ms
[50] div+add │ /encoder/layer.4/attention/self/MatMul
t1 fused : 0.005568 ms
t2 separate : 0.007040 ms (Δ 0.001472 ms, speed-up 1.26×)
── t2 breakdown ──
normal : 0.004544 ms
Div : 0.002496 ms
[51] double_add │ /encoder/layer.4/attention/output/dense/MatMul
t1 fused : 0.023841 ms
t2 separate : 0.026433 ms (Δ 0.002592 ms, speed-up 1.11×)
── t2 breakdown ──
normal : 0.022881 ms
Add : 0.002336 ms
Add : 0.001216 ms
[52] sub_fusion │ /encoder/layer.4/attention/output/LayerNorm/ReduceMean
t1 fused : 0.003104 ms
t2 separate : 0.004096 ms (Δ 0.000992 ms, speed-up 1.32×)
── t2 breakdown ──
normal : 0.002144 ms
Sub : 0.001952 ms
[53] normalization_fusion │ /encoder/layer.4/attention/output/LayerNorm/ReduceMean_1
t1 fused : 0.004032 ms
t2 separate : 0.011201 ms (Δ 0.007169 ms, speed-up 2.78×)
── t2 breakdown ──
normal : 0.002144 ms
Add : 0.001153 ms
Sqrt : 0.000928 ms
Div : 0.002336 ms
Mul : 0.002272 ms
Add : 0.002368 ms
[54] auto_fusion │ /encoder/layer.4/intermediate/dense/MatMul
t1 fused : 0.034689 ms
t2 separate : 0.040737 ms (Δ 0.006048 ms, speed-up 1.17×)
── t2 breakdown ──
normal : 0.023841 ms
Add : 0.003552 ms
Div : 0.003808 ms
Erf : 0.002048 ms
Add : 0.002816 ms
Mul : 0.001856 ms
Mul : 0.002816 ms
[55] double_add │ /encoder/layer.4/output/dense/MatMul
t1 fused : 0.083490 ms
t2 separate : 0.086146 ms (Δ 0.002656 ms, speed-up 1.03×)
── t2 breakdown ──
normal : 0.082562 ms
Add : 0.002336 ms
Add : 0.001248 ms
[56] sub_fusion │ /encoder/layer.4/output/LayerNorm/ReduceMean
t1 fused : 0.003104 ms
t2 separate : 0.004128 ms (Δ 0.001024 ms, speed-up 1.33×)
── t2 breakdown ──
normal : 0.002176 ms
Sub : 0.001952 ms
[57] normalization_fusion │ /encoder/layer.4/output/LayerNorm/ReduceMean_1
t1 fused : 0.004001 ms
t2 separate : 0.011231 ms (Δ 0.007230 ms, speed-up 2.81×)
── t2 breakdown ──
normal : 0.002144 ms
Add : 0.001151 ms
Sqrt : 0.000896 ms
Div : 0.002304 ms
Mul : 0.002336 ms
Add : 0.002400 ms
[58] Add │ /encoder/layer.5/attention/self/query/MatMul
t1 fused : 0.023425 ms
t2 separate : 0.025185 ms (Δ 0.001760 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022881 ms
Add : 0.002304 ms
[59] Add │ /encoder/layer.5/attention/self/key/MatMul
t1 fused : 0.023520 ms
t2 separate : 0.025216 ms (Δ 0.001696 ms, speed-up 1.07×)
── t2 breakdown ──
normal : 0.022880 ms
Add : 0.002336 ms
[60] Add │ /encoder/layer.5/attention/self/value/MatMul
t1 fused : 0.023616 ms
t2 separate : 0.025122 ms (Δ 0.001506 ms, speed-up 1.06×)
── t2 breakdown ──
normal : 0.022849 ms
Add : 0.002273 ms
[61] div+add │ /encoder/layer.5/attention/self/MatMul
t1 fused : 0.005568 ms
t2 separate : 0.007040 ms (Δ 0.001472 ms, speed-up 1.26×)
── t2 breakdown ──
normal : 0.004544 ms
Div : 0.002496 ms
[62] double_add │ /encoder/layer.5/attention/output/dense/MatMul
t1 fused : 0.023648 ms
t2 separate : 0.026370 ms (Δ 0.002722 ms, speed-up 1.12×)
── t2 breakdown ──
normal : 0.022913 ms
Add : 0.002240 ms
Add : 0.001217 ms
[63] sub_fusion │ /encoder/layer.5/attention/output/LayerNorm/ReduceMean
t1 fused : 0.003104 ms
t2 separate : 0.004096 ms (Δ 0.000992 ms, speed-up 1.32×)
── t2 breakdown ──
normal : 0.002144 ms
Sub : 0.001952 ms
[64] normalization_fusion │ /encoder/layer.5/attention/output/LayerNorm/ReduceMean_1
t1 fused : 0.004033 ms
t2 separate : 0.011360 ms (Δ 0.007327 ms, speed-up 2.82×)
── t2 breakdown ──
normal : 0.002112 ms
Add : 0.001152 ms
Sqrt : 0.000928 ms
Div : 0.002304 ms
Mul : 0.002368 ms
Add : 0.002496 ms
[65] auto_fusion │ /encoder/layer.5/intermediate/dense/MatMul
t1 fused : 0.034657 ms
t2 separate : 0.041344 ms (Δ 0.006687 ms, speed-up 1.19×)
── t2 breakdown ──
normal : 0.024064 ms
Add : 0.003872 ms
Div : 0.003808 ms
Erf : 0.002080 ms
Add : 0.002848 ms
Mul : 0.001824 ms
Mul : 0.002848 ms
[66] double_add │ /encoder/layer.5/output/dense/MatMul
t1 fused : 0.083490 ms
t2 separate : 0.086178 ms (Δ 0.002688 ms, speed-up 1.03×)
── t2 breakdown ──
normal : 0.082626 ms
Add : 0.002304 ms
Add : 0.001248 ms
[67] sub_fusion │ /encoder/layer.5/output/LayerNorm/ReduceMean
t1 fused : 0.003104 ms
t2 separate : 0.004160 ms (Δ 0.001056 ms, speed-up 1.34×)
── t2 breakdown ──
normal : 0.002208 ms
Sub : 0.001952 ms
[68] normalization_fusion │ /encoder/layer.5/output/LayerNorm/ReduceMean_1
t1 fused : 0.004000 ms
t2 separate : 0.011136 ms (Δ 0.007136 ms, speed-up 2.78×)
── t2 breakdown ──
normal : 0.002112 ms
Add : 0.001152 ms
Sqrt : 0.000896 ms
Div : 0.002336 ms
Mul : 0.002272 ms
Add : 0.002368 ms
[69] Add │ /encoder/layer.6/attention/self/query/MatMul
t1 fused : 0.023425 ms
t2 separate : 0.025186 ms (Δ 0.001761 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022945 ms
Add : 0.002241 ms
[70] Add │ /encoder/layer.6/attention/self/key/MatMul
t1 fused : 0.023425 ms
t2 separate : 0.025185 ms (Δ 0.001760 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022881 ms
Add : 0.002304 ms
[71] Add │ /encoder/layer.6/attention/self/value/MatMul
t1 fused : 0.023425 ms
t2 separate : 0.025121 ms (Δ 0.001696 ms, speed-up 1.07×)
── t2 breakdown ──
normal : 0.022881 ms
Add : 0.002240 ms
[72] div+add │ /encoder/layer.6/attention/self/MatMul
t1 fused : 0.005568 ms
t2 separate : 0.007041 ms (Δ 0.001473 ms, speed-up 1.26×)
── t2 breakdown ──
normal : 0.004576 ms
Div : 0.002465 ms
[73] double_add │ /encoder/layer.6/attention/output/dense/MatMul
t1 fused : 0.023712 ms
t2 separate : 0.026752 ms (Δ 0.003040 ms, speed-up 1.13×)
── t2 breakdown ──
normal : 0.023232 ms
Add : 0.002304 ms
Add : 0.001216 ms
[74] sub_fusion │ /encoder/layer.6/attention/output/LayerNorm/ReduceMean
t1 fused : 0.003072 ms
t2 separate : 0.004096 ms (Δ 0.001024 ms, speed-up 1.33×)
── t2 breakdown ──
normal : 0.002144 ms
Sub : 0.001952 ms
[75] normalization_fusion │ /encoder/layer.6/attention/output/LayerNorm/ReduceMean_1
t1 fused : 0.004032 ms
t2 separate : 0.011137 ms (Δ 0.007105 ms, speed-up 2.76×)
── t2 breakdown ──
normal : 0.002144 ms
Add : 0.001152 ms
Sqrt : 0.000896 ms
Div : 0.002336 ms
Mul : 0.002304 ms
Add : 0.002305 ms
[76] auto_fusion │ /encoder/layer.6/intermediate/dense/MatMul
t1 fused : 0.034593 ms
t2 separate : 0.041185 ms (Δ 0.006592 ms, speed-up 1.19×)
── t2 breakdown ──
normal : 0.023873 ms
Add : 0.003872 ms
Div : 0.003840 ms
Erf : 0.002080 ms
Add : 0.002816 ms
Mul : 0.001888 ms
Mul : 0.002816 ms
[77] double_add │ /encoder/layer.6/output/dense/MatMul
t1 fused : 0.083426 ms
t2 separate : 0.086146 ms (Δ 0.002720 ms, speed-up 1.03×)
── t2 breakdown ──
normal : 0.082626 ms
Add : 0.002304 ms
Add : 0.001216 ms
[78] sub_fusion │ /encoder/layer.6/output/LayerNorm/ReduceMean
t1 fused : 0.003104 ms
t2 separate : 0.004128 ms (Δ 0.001024 ms, speed-up 1.33×)
── t2 breakdown ──
normal : 0.002176 ms
Sub : 0.001952 ms
[79] normalization_fusion │ /encoder/layer.6/output/LayerNorm/ReduceMean_1
t1 fused : 0.004000 ms
t2 separate : 0.011073 ms (Δ 0.007073 ms, speed-up 2.77×)
── t2 breakdown ──
normal : 0.002144 ms
Add : 0.001152 ms
Sqrt : 0.000896 ms
Div : 0.002337 ms
Mul : 0.002304 ms
Add : 0.002240 ms
[80] Add │ /encoder/layer.7/attention/self/query/MatMul
t1 fused : 0.023425 ms
t2 separate : 0.025216 ms (Δ 0.001791 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022880 ms
Add : 0.002336 ms
[81] Add │ /encoder/layer.7/attention/self/key/MatMul
t1 fused : 0.023585 ms
t2 separate : 0.025089 ms (Δ 0.001504 ms, speed-up 1.06×)
── t2 breakdown ──
normal : 0.022849 ms
Add : 0.002240 ms
[82] Add │ /encoder/layer.7/attention/self/value/MatMul
t1 fused : 0.023361 ms
t2 separate : 0.025153 ms (Δ 0.001792 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022881 ms
Add : 0.002272 ms
[83] div+add │ /encoder/layer.7/attention/self/MatMul
t1 fused : 0.005600 ms
t2 separate : 0.007041 ms (Δ 0.001441 ms, speed-up 1.26×)
── t2 breakdown ──
normal : 0.004576 ms
Div : 0.002465 ms
[84] double_add │ /encoder/layer.7/attention/output/dense/MatMul
t1 fused : 0.023777 ms
t2 separate : 0.026529 ms (Δ 0.002752 ms, speed-up 1.12×)
── t2 breakdown ──
normal : 0.023041 ms
Add : 0.002240 ms
Add : 0.001248 ms
[85] sub_fusion │ /encoder/layer.7/attention/output/LayerNorm/ReduceMean
t1 fused : 0.003072 ms
t2 separate : 0.004064 ms (Δ 0.000992 ms, speed-up 1.32×)
── t2 breakdown ──
normal : 0.002112 ms
Sub : 0.001952 ms
[86] normalization_fusion │ /encoder/layer.7/attention/output/LayerNorm/ReduceMean_1
t1 fused : 0.004032 ms
t2 separate : 0.011136 ms (Δ 0.007104 ms, speed-up 2.76×)
── t2 breakdown ──
normal : 0.002144 ms
Add : 0.001152 ms
Sqrt : 0.000928 ms
Div : 0.002304 ms
Mul : 0.002304 ms
Add : 0.002304 ms
[87] auto_fusion │ /encoder/layer.7/intermediate/dense/MatMul
t1 fused : 0.034817 ms
t2 separate : 0.041857 ms (Δ 0.007040 ms, speed-up 1.20×)
── t2 breakdown ──
normal : 0.024512 ms
Add : 0.003968 ms
Div : 0.003809 ms
Erf : 0.002048 ms
Add : 0.002816 ms
Mul : 0.001856 ms
Mul : 0.002848 ms
[88] double_add │ /encoder/layer.7/output/dense/MatMul
t1 fused : 0.083426 ms
t2 separate : 0.085986 ms (Δ 0.002560 ms, speed-up 1.03×)
── t2 breakdown ──
normal : 0.082594 ms
Add : 0.002208 ms
Add : 0.001184 ms
[89] sub_fusion │ /encoder/layer.7/output/LayerNorm/ReduceMean
t1 fused : 0.003072 ms
t2 separate : 0.004192 ms (Δ 0.001120 ms, speed-up 1.36×)
── t2 breakdown ──
normal : 0.002240 ms
Sub : 0.001952 ms
[90] normalization_fusion │ /encoder/layer.7/output/LayerNorm/ReduceMean_1
t1 fused : 0.004000 ms
t2 separate : 0.011104 ms (Δ 0.007104 ms, speed-up 2.78×)
── t2 breakdown ──
normal : 0.002112 ms
Add : 0.001152 ms
Sqrt : 0.000896 ms
Div : 0.002336 ms
Mul : 0.002304 ms
Add : 0.002304 ms
[91] Add │ /encoder/layer.8/attention/self/query/MatMul
t1 fused : 0.023521 ms
t2 separate : 0.025216 ms (Δ 0.001695 ms, speed-up 1.07×)
── t2 breakdown ──
normal : 0.022848 ms
Add : 0.002368 ms
[92] Add │ /encoder/layer.8/attention/self/key/MatMul
t1 fused : 0.023521 ms
t2 separate : 0.025185 ms (Δ 0.001664 ms, speed-up 1.07×)
── t2 breakdown ──
normal : 0.022881 ms
Add : 0.002304 ms
[93] Add │ /encoder/layer.8/attention/self/value/MatMul
t1 fused : 0.023393 ms
t2 separate : 0.025345 ms (Δ 0.001952 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022912 ms
Add : 0.002433 ms
[94] div+add │ /encoder/layer.8/attention/self/MatMul
t1 fused : 0.005569 ms
t2 separate : 0.007104 ms (Δ 0.001535 ms, speed-up 1.28×)
── t2 breakdown ──
normal : 0.004608 ms
Div : 0.002496 ms
[95] double_add │ /encoder/layer.8/attention/output/dense/MatMul
t1 fused : 0.023712 ms
t2 separate : 0.026593 ms (Δ 0.002881 ms, speed-up 1.12×)
── t2 breakdown ──
normal : 0.023073 ms
Add : 0.002336 ms
Add : 0.001184 ms
[96] sub_fusion │ /encoder/layer.8/attention/output/LayerNorm/ReduceMean
t1 fused : 0.003104 ms
t2 separate : 0.004095 ms (Δ 0.000991 ms, speed-up 1.32×)
── t2 breakdown ──
normal : 0.002144 ms
Sub : 0.001951 ms
[97] normalization_fusion │ /encoder/layer.8/attention/output/LayerNorm/ReduceMean_1
t1 fused : 0.004032 ms
t2 separate : 0.011298 ms (Δ 0.007266 ms, speed-up 2.80×)
── t2 breakdown ──
normal : 0.002144 ms
Add : 0.001152 ms
Sqrt : 0.000929 ms
Div : 0.002304 ms
Mul : 0.002432 ms
Add : 0.002337 ms
[98] auto_fusion │ /encoder/layer.8/intermediate/dense/MatMul
t1 fused : 0.034753 ms
t2 separate : 0.041761 ms (Δ 0.007008 ms, speed-up 1.20×)
── t2 breakdown ──
normal : 0.024513 ms
Add : 0.003872 ms
Div : 0.003840 ms
Erf : 0.002048 ms
Add : 0.002848 ms
Mul : 0.001792 ms
Mul : 0.002848 ms
[99] double_add │ /encoder/layer.8/output/dense/MatMul
t1 fused : 0.083426 ms
t2 separate : 0.086274 ms (Δ 0.002848 ms, speed-up 1.03×)
── t2 breakdown ──
normal : 0.082690 ms
Add : 0.002368 ms
Add : 0.001216 ms
[100] sub_fusion │ /encoder/layer.8/output/LayerNorm/ReduceMean
t1 fused : 0.003072 ms
t2 separate : 0.004128 ms (Δ 0.001056 ms, speed-up 1.34×)
── t2 breakdown ──
normal : 0.002208 ms
Sub : 0.001920 ms
[101] normalization_fusion │ /encoder/layer.8/output/LayerNorm/ReduceMean_1
t1 fused : 0.004033 ms
t2 separate : 0.011200 ms (Δ 0.007167 ms, speed-up 2.78×)
── t2 breakdown ──
normal : 0.002176 ms
Add : 0.001152 ms
Sqrt : 0.000896 ms
Div : 0.002336 ms
Mul : 0.002368 ms
Add : 0.002272 ms
[102] Add │ /encoder/layer.9/attention/self/query/MatMul
t1 fused : 0.023424 ms
t2 separate : 0.025281 ms (Δ 0.001857 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022977 ms
Add : 0.002304 ms
[103] Add │ /encoder/layer.9/attention/self/key/MatMul
t1 fused : 0.023392 ms
t2 separate : 0.025184 ms (Δ 0.001792 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022880 ms
Add : 0.002304 ms
[104] Add │ /encoder/layer.9/attention/self/value/MatMul
t1 fused : 0.023393 ms
t2 separate : 0.025185 ms (Δ 0.001792 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022945 ms
Add : 0.002240 ms
[105] div+add │ /encoder/layer.9/attention/self/MatMul
t1 fused : 0.005568 ms
t2 separate : 0.007073 ms (Δ 0.001505 ms, speed-up 1.27×)
── t2 breakdown ──
normal : 0.004577 ms
Div : 0.002496 ms
[106] double_add │ /encoder/layer.9/attention/output/dense/MatMul
t1 fused : 0.023713 ms
t2 separate : 0.026400 ms (Δ 0.002687 ms, speed-up 1.11×)
── t2 breakdown ──
normal : 0.022880 ms
Add : 0.002304 ms
Add : 0.001216 ms
[107] sub_fusion │ /encoder/layer.9/attention/output/LayerNorm/ReduceMean
t1 fused : 0.003072 ms
t2 separate : 0.004065 ms (Δ 0.000993 ms, speed-up 1.32×)
── t2 breakdown ──
normal : 0.002113 ms
Sub : 0.001952 ms
[108] normalization_fusion │ /encoder/layer.9/attention/output/LayerNorm/ReduceMean_1
t1 fused : 0.004000 ms
t2 separate : 0.011201 ms (Δ 0.007201 ms, speed-up 2.80×)
── t2 breakdown ──
normal : 0.002112 ms
Add : 0.001153 ms
Sqrt : 0.000928 ms
Div : 0.002304 ms
Mul : 0.002336 ms
Add : 0.002368 ms
[109] auto_fusion │ /encoder/layer.9/intermediate/dense/MatMul
t1 fused : 0.034849 ms
t2 separate : 0.041185 ms (Δ 0.006336 ms, speed-up 1.18×)
── t2 breakdown ──
normal : 0.023873 ms
Add : 0.003872 ms
Div : 0.003872 ms
Erf : 0.002048 ms
Add : 0.002848 ms
Mul : 0.001824 ms
Mul : 0.002848 ms
[110] double_add │ /encoder/layer.9/output/dense/MatMul
t1 fused : 0.083426 ms
t2 separate : 0.086081 ms (Δ 0.002655 ms, speed-up 1.03×)
── t2 breakdown ──
normal : 0.082626 ms
Add : 0.002271 ms
Add : 0.001184 ms
[111] sub_fusion │ /encoder/layer.9/output/LayerNorm/ReduceMean
t1 fused : 0.003072 ms
t2 separate : 0.004160 ms (Δ 0.001088 ms, speed-up 1.35×)
── t2 breakdown ──
normal : 0.002208 ms
Sub : 0.001952 ms
[112] normalization_fusion │ /encoder/layer.9/output/LayerNorm/ReduceMean_1
t1 fused : 0.004032 ms
t2 separate : 0.011040 ms (Δ 0.007008 ms, speed-up 2.74×)
── t2 breakdown ──
normal : 0.002112 ms
Add : 0.001152 ms
Sqrt : 0.000896 ms
Div : 0.002336 ms
Mul : 0.002240 ms
Add : 0.002304 ms
[113] Add │ /encoder/layer.10/attention/self/query/MatMul
t1 fused : 0.023392 ms
t2 separate : 0.025184 ms (Δ 0.001792 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022880 ms
Add : 0.002304 ms
[114] Add │ /encoder/layer.10/attention/self/key/MatMul
t1 fused : 0.023424 ms
t2 separate : 0.025281 ms (Δ 0.001857 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022881 ms
Add : 0.002400 ms
[115] Add │ /encoder/layer.10/attention/self/value/MatMul
t1 fused : 0.023681 ms
t2 separate : 0.025281 ms (Δ 0.001600 ms, speed-up 1.07×)
── t2 breakdown ──
normal : 0.022913 ms
Add : 0.002368 ms
[116] div+add │ /encoder/layer.10/attention/self/MatMul
t1 fused : 0.005568 ms
t2 separate : 0.007008 ms (Δ 0.001440 ms, speed-up 1.26×)
── t2 breakdown ──
normal : 0.004544 ms
Div : 0.002464 ms
[117] double_add │ /encoder/layer.10/attention/output/dense/MatMul
t1 fused : 0.023649 ms
t2 separate : 0.026336 ms (Δ 0.002687 ms, speed-up 1.11×)
── t2 breakdown ──
normal : 0.022849 ms
Add : 0.002272 ms
Add : 0.001215 ms
[118] sub_fusion │ /encoder/layer.10/attention/output/LayerNorm/ReduceMean
t1 fused : 0.003104 ms
t2 separate : 0.004096 ms (Δ 0.000992 ms, speed-up 1.32×)
── t2 breakdown ──
normal : 0.002144 ms
Sub : 0.001952 ms
[119] normalization_fusion │ /encoder/layer.10/attention/output/LayerNorm/ReduceMean_1
t1 fused : 0.004032 ms
t2 separate : 0.011168 ms (Δ 0.007136 ms, speed-up 2.77×)
── t2 breakdown ──
normal : 0.002144 ms
Add : 0.001184 ms
Sqrt : 0.000928 ms
Div : 0.002304 ms
Mul : 0.002304 ms
Add : 0.002304 ms
[120] auto_fusion │ /encoder/layer.10/intermediate/dense/MatMul
t1 fused : 0.034753 ms
t2 separate : 0.041345 ms (Δ 0.006592 ms, speed-up 1.19×)
── t2 breakdown ──
normal : 0.024193 ms
Add : 0.003776 ms
Div : 0.003808 ms
Erf : 0.002048 ms
Add : 0.002816 ms
Mul : 0.001888 ms
Mul : 0.002816 ms
[121] double_add │ /encoder/layer.10/output/dense/MatMul
t1 fused : 0.083490 ms
t2 separate : 0.086147 ms (Δ 0.002657 ms, speed-up 1.03×)
── t2 breakdown ──
normal : 0.082626 ms
Add : 0.002305 ms
Add : 0.001216 ms
[122] sub_fusion │ /encoder/layer.10/output/LayerNorm/ReduceMean
t1 fused : 0.003104 ms
t2 separate : 0.004128 ms (Δ 0.001024 ms, speed-up 1.33×)
── t2 breakdown ──
normal : 0.002208 ms
Sub : 0.001920 ms
[123] normalization_fusion │ /encoder/layer.10/output/LayerNorm/ReduceMean_1
t1 fused : 0.004000 ms
t2 separate : 0.011104 ms (Δ 0.007104 ms, speed-up 2.78×)
── t2 breakdown ──
normal : 0.002144 ms
Add : 0.001152 ms
Sqrt : 0.000896 ms
Div : 0.002304 ms
Mul : 0.002304 ms
Add : 0.002304 ms
[124] Add │ /encoder/layer.11/attention/self/query/MatMul
t1 fused : 0.023425 ms
t2 separate : 0.025185 ms (Δ 0.001760 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022849 ms
Add : 0.002336 ms
[125] Add │ /encoder/layer.11/attention/self/key/MatMul
t1 fused : 0.023392 ms
t2 separate : 0.025249 ms (Δ 0.001857 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022881 ms
Add : 0.002368 ms
[126] Add │ /encoder/layer.11/attention/self/value/MatMul
t1 fused : 0.023392 ms
t2 separate : 0.025152 ms (Δ 0.001760 ms, speed-up 1.08×)
── t2 breakdown ──
normal : 0.022881 ms
Add : 0.002271 ms
[127] div+add │ /encoder/layer.11/attention/self/MatMul
t1 fused : 0.005568 ms
t2 separate : 0.007008 ms (Δ 0.001440 ms, speed-up 1.26×)
── t2 breakdown ──
normal : 0.004544 ms
Div : 0.002464 ms
[128] double_add │ /encoder/layer.11/attention/output/dense/MatMul
t1 fused : 0.023777 ms
t2 separate : 0.026432 ms (Δ 0.002655 ms, speed-up 1.11×)
── t2 breakdown ──
normal : 0.022912 ms
Add : 0.002304 ms
Add : 0.001216 ms
[129] sub_fusion │ /encoder/layer.11/attention/output/LayerNorm/ReduceMean
t1 fused : 0.003072 ms
t2 separate : 0.004032 ms (Δ 0.000960 ms, speed-up 1.31×)
── t2 breakdown ──
normal : 0.002112 ms
Sub : 0.001920 ms
[130] normalization_fusion │ /encoder/layer.11/attention/output/LayerNorm/ReduceMean_1
t1 fused : 0.004000 ms
t2 separate : 0.011200 ms (Δ 0.007200 ms, speed-up 2.80×)
── t2 breakdown ──
normal : 0.002176 ms
Add : 0.001152 ms
Sqrt : 0.000928 ms
Div : 0.002336 ms
Mul : 0.002208 ms
Add : 0.002400 ms
[131] auto_fusion │ /encoder/layer.11/intermediate/dense/MatMul
t1 fused : 0.034817 ms
t2 separate : 0.041088 ms (Δ 0.006271 ms, speed-up 1.18×)
── t2 breakdown ──
normal : 0.023904 ms
Add : 0.003808 ms
Div : 0.003840 ms
Erf : 0.002016 ms
Add : 0.002816 ms
Mul : 0.001856 ms
Mul : 0.002848 ms
[132] double_add │ /encoder/layer.11/output/dense/MatMul
t1 fused : 0.083427 ms
t2 separate : 0.086114 ms (Δ 0.002687 ms, speed-up 1.03×)
── t2 breakdown ──
normal : 0.082594 ms
Add : 0.002304 ms
Add : 0.001216 ms
[133] sub_fusion │ /encoder/layer.11/output/LayerNorm/ReduceMean
t1 fused : 0.003072 ms
t2 separate : 0.004160 ms (Δ 0.001088 ms, speed-up 1.35×)
── t2 breakdown ──
normal : 0.002208 ms
Sub : 0.001952 ms
[134] normalization_fusion │ /encoder/layer.11/output/LayerNorm/ReduceMean_1
t1 fused : 0.004000 ms
t2 separate : 0.011202 ms (Δ 0.007202 ms, speed-up 2.80×)
── t2 breakdown ──
normal : 0.002145 ms
Add : 0.001152 ms
Sqrt : 0.000896 ms
Div : 0.002304 ms
Mul : 0.002337 ms
Add : 0.002368 ms
===================================================================================================================
► 每個『組合類型』總和
sub_fusion : fused 0.077346 ms sep 0.102752 ms Δ 0.025406 ms speed-up 1.33×
normalization_fusion : fused 0.100419 ms sep 0.278862 ms Δ 0.178443 ms speed-up 2.78×
Add : fused 0.843893 ms sep 0.907709 ms Δ 0.063816 ms speed-up 1.08×
div+add : fused 0.066882 ms sep 0.084548 ms Δ 0.017666 ms speed-up 1.26×
double_add : fused 1.285888 ms sep 1.351396 ms Δ 0.065508 ms speed-up 1.05×
auto_fusion : fused 0.416717 ms sep 0.494731 ms Δ 0.078014 ms speed-up 1.19×
===================================================================================================================
► 四組合累計 Σ
Σ fused : 2.791145 ms
Σ separate : 3.219998 ms
Δ (sep-fuse): 0.428853 ms speed-up 1.15×
===================================================================================================================
► 其餘 Run_with_type 彙總 (未納入組合)
Add : t1 0.004736 ms │ t2 0.031775 ms │ Δ 0.027039 ms
Cast : t1 0.001248 ms │ t2 0.001280 ms │ Δ 0.000032 ms
CumSum : t1 0.021569 ms │ t2 0.021537 ms │ Δ -0.000032 ms
Equal : t1 0.001473 ms │ t2 0.001440 ms │ Δ -0.000033 ms
Gather : t1 0.076227 ms │ t2 0.082146 ms │ Δ 0.005919 ms
Gemm : t1 0.035297 ms │ t2 0.032833 ms │ Δ -0.002464 ms
Mul : t1 0.002625 ms │ t2 0.002751 ms │ Δ 0.000126 ms
Not : t1 0.000960 ms │ t2 0.000960 ms │ Δ 0.000000 ms
Pow : t1 0.035649 ms │ t2 0.035521 ms │ Δ -0.000128 ms
Softmax : t1 0.031200 ms │ t2 0.031268 ms │ Δ 0.000068 ms
Sub : t1 0.001184 ms │ t2 0.001152 ms │ Δ -0.000032 ms
Tanh : t1 0.000960 ms │ t2 0.000961 ms │ Δ 0.000001 ms
Transpose : t1 0.099461 ms │ t2 0.100450 ms │ Δ 0.000989 ms
===================================================================================================================
► 子-op (含 normal) 跨全部組合累積時間
normal : 2.541801 ms
Add : 0.334635 ms
Div : 0.133669 ms
Mul : 0.113892 ms
Sub : 0.048640 ms
Erf : 0.024608 ms
Sqrt : 0.022753 ms
===================================================================================================================
► 整體檔案總執行時間
t1 total : 3.177336 ms
t2 total : 3.637417 ms
Δ : 0.460081 ms speed-up 1.14×
Editor is loading...
Leave a Comment