Untitled

 avatar
unknown
c_cpp
3 days ago
75 kB
8
Indexable
šŸ”„ Starting from: MatMul : bert/embeddings/MatMul
   ā›” Encountered non-elementwise: Reshape : bert/embeddings/Reshape_3
āš ļø No fusable chain found.
šŸ”„ Starting from: Conv : bert/embeddings/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/embeddings/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/embeddings/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/embeddings/LayerNorm/batchnorm/Rsqrt__74
   āœ… Fusable: Mul : bert/embeddings/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/embeddings/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 0:
   + Conv : bert/embeddings/LayerNorm/moments/variance/conv
   + Add : bert/embeddings/LayerNorm/batchnorm/add
   + Sqrt : bert/embeddings/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/embeddings/LayerNorm/batchnorm/Rsqrt__74
   + Mul : bert/embeddings/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_0/attention/self/value/MatMul
   āœ… Fusable: Add : bert/encoder/layer_0/attention/self/value/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_0/attention/self/Reshape_2
🟢 Final fusable group 1:
   + MatMul : bert/encoder/layer_0/attention/self/value/MatMul
   + Add : bert/encoder/layer_0/attention/self/value/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_0/attention/self/query/MatMul
   āœ… Fusable: Add : bert/encoder/layer_0/attention/self/query/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_0/attention/self/Reshape
🟢 Final fusable group 2:
   + MatMul : bert/encoder/layer_0/attention/self/query/MatMul
   + Add : bert/encoder/layer_0/attention/self/query/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_0/attention/self/key/MatMul
   āœ… Fusable: Add : bert/encoder/layer_0/attention/self/key/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_0/attention/self/Reshape_1
🟢 Final fusable group 3:
   + MatMul : bert/encoder/layer_0/attention/self/key/MatMul
   + Add : bert/encoder/layer_0/attention/self/key/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_0/attention/self/MatMul
   āœ… Fusable: Mul : bert/encoder/layer_0/attention/self/Mul
   āœ… Fusable: Add : bert/encoder/layer_0/attention/self/add
   ā›” Encountered non-elementwise: Softmax : bert/encoder/layer_0/attention/self/Softmax
🟢 Final fusable group 4:
   + MatMul : bert/encoder/layer_0/attention/self/MatMul
   + Mul : bert/encoder/layer_0/attention/self/Mul
   + Add : bert/encoder/layer_0/attention/self/add
šŸ”„ Starting from: MatMul : bert/encoder/layer_0/attention/self/MatMul_1
   ā›” Encountered non-elementwise: Transpose : bert/encoder/layer_0/attention/self/transpose_3
āš ļø No fusable chain found.
šŸ”„ Starting from: MatMul : bert/encoder/layer_0/attention/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_0/attention/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_0/attention/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_0/attention/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 5:
   + MatMul : bert/encoder/layer_0/attention/output/dense/MatMul
   + Add : bert/encoder/layer_0/attention/output/dense/BiasAdd
   + Add : bert/encoder/layer_0/attention/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_0/attention/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/Rsqrt__311
   āœ… Fusable: Mul : bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_0/attention/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 6:
   + Conv : bert/encoder/layer_0/attention/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/Rsqrt__311
   + Mul : bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_0/intermediate/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_0/intermediate/dense/BiasAdd
šŸ” Exploring multi-output node: Add : bert/encoder/layer_0/intermediate/dense/BiasAdd
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_0/intermediate/dense/mul_3
   āœ… Fusable: Mul : bert/encoder/layer_0/intermediate/dense/mul_3
   ā›” Encountered non-elementwise: MatMul : bert/encoder/layer_0/output/dense/MatMul
🟢 Final fusable group 7:
   + MatMul : bert/encoder/layer_0/intermediate/dense/MatMul
   + Add : bert/encoder/layer_0/intermediate/dense/BiasAdd
   + Mul : bert/encoder/layer_0/intermediate/dense/mul_2
   + Add : bert/encoder/layer_0/intermediate/dense/add_1
   + Tanh : bert/encoder/layer_0/intermediate/dense/Tanh
   + Mul : bert/encoder/layer_0/intermediate/dense/mul_1
   + Mul : bert/encoder/layer_0/intermediate/dense/mul
   + Add : bert/encoder/layer_0/intermediate/dense/add
   + Pow : bert/encoder/layer_0/intermediate/dense/Pow
   + Mul : bert/encoder/layer_0/intermediate/dense/mul_3
šŸ”„ Starting from: MatMul : bert/encoder/layer_0/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_0/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_0/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_0/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_0/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 8:
   + MatMul : bert/encoder/layer_0/output/dense/MatMul
   + Add : bert/encoder/layer_0/output/dense/BiasAdd
   + Add : bert/encoder/layer_0/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_0/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_0/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_0/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_0/output/LayerNorm/batchnorm/Rsqrt__315
   āœ… Fusable: Mul : bert/encoder/layer_0/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_0/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 9:
   + Conv : bert/encoder/layer_0/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_0/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_0/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_0/output/LayerNorm/batchnorm/Rsqrt__315
   + Mul : bert/encoder/layer_0/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_1/attention/self/value/MatMul
   āœ… Fusable: Add : bert/encoder/layer_1/attention/self/value/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_1/attention/self/Reshape_2
🟢 Final fusable group 10:
   + MatMul : bert/encoder/layer_1/attention/self/value/MatMul
   + Add : bert/encoder/layer_1/attention/self/value/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_1/attention/self/query/MatMul
   āœ… Fusable: Add : bert/encoder/layer_1/attention/self/query/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_1/attention/self/Reshape
🟢 Final fusable group 11:
   + MatMul : bert/encoder/layer_1/attention/self/query/MatMul
   + Add : bert/encoder/layer_1/attention/self/query/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_1/attention/self/key/MatMul
   āœ… Fusable: Add : bert/encoder/layer_1/attention/self/key/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_1/attention/self/Reshape_1
🟢 Final fusable group 12:
   + MatMul : bert/encoder/layer_1/attention/self/key/MatMul
   + Add : bert/encoder/layer_1/attention/self/key/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_1/attention/self/MatMul
   āœ… Fusable: Mul : bert/encoder/layer_1/attention/self/Mul
   āœ… Fusable: Add : bert/encoder/layer_1/attention/self/add
   ā›” Encountered non-elementwise: Softmax : bert/encoder/layer_1/attention/self/Softmax
🟢 Final fusable group 13:
   + MatMul : bert/encoder/layer_1/attention/self/MatMul
   + Mul : bert/encoder/layer_1/attention/self/Mul
   + Add : bert/encoder/layer_1/attention/self/add
šŸ”„ Starting from: MatMul : bert/encoder/layer_1/attention/self/MatMul_1
   ā›” Encountered non-elementwise: Transpose : bert/encoder/layer_1/attention/self/transpose_3
āš ļø No fusable chain found.
šŸ”„ Starting from: MatMul : bert/encoder/layer_1/attention/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_1/attention/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_1/attention/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_1/attention/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 14:
   + MatMul : bert/encoder/layer_1/attention/output/dense/MatMul
   + Add : bert/encoder/layer_1/attention/output/dense/BiasAdd
   + Add : bert/encoder/layer_1/attention/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_1/attention/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/Rsqrt__325
   āœ… Fusable: Mul : bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_1/attention/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 15:
   + Conv : bert/encoder/layer_1/attention/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/Rsqrt__325
   + Mul : bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_1/intermediate/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_1/intermediate/dense/BiasAdd
šŸ” Exploring multi-output node: Add : bert/encoder/layer_1/intermediate/dense/BiasAdd
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_1/intermediate/dense/mul_3
   āœ… Fusable: Mul : bert/encoder/layer_1/intermediate/dense/mul_3
   ā›” Encountered non-elementwise: MatMul : bert/encoder/layer_1/output/dense/MatMul
🟢 Final fusable group 16:
   + MatMul : bert/encoder/layer_1/intermediate/dense/MatMul
   + Add : bert/encoder/layer_1/intermediate/dense/BiasAdd
   + Mul : bert/encoder/layer_1/intermediate/dense/mul_2
   + Add : bert/encoder/layer_1/intermediate/dense/add_1
   + Tanh : bert/encoder/layer_1/intermediate/dense/Tanh
   + Mul : bert/encoder/layer_1/intermediate/dense/mul_1
   + Mul : bert/encoder/layer_1/intermediate/dense/mul
   + Add : bert/encoder/layer_1/intermediate/dense/add
   + Pow : bert/encoder/layer_1/intermediate/dense/Pow
   + Mul : bert/encoder/layer_1/intermediate/dense/mul_3
šŸ”„ Starting from: MatMul : bert/encoder/layer_1/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_1/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_1/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_1/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_1/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 17:
   + MatMul : bert/encoder/layer_1/output/dense/MatMul
   + Add : bert/encoder/layer_1/output/dense/BiasAdd
   + Add : bert/encoder/layer_1/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_1/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_1/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_1/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_1/output/LayerNorm/batchnorm/Rsqrt__329
   āœ… Fusable: Mul : bert/encoder/layer_1/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_1/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 18:
   + Conv : bert/encoder/layer_1/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_1/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_1/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_1/output/LayerNorm/batchnorm/Rsqrt__329
   + Mul : bert/encoder/layer_1/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_2/attention/self/value/MatMul
   āœ… Fusable: Add : bert/encoder/layer_2/attention/self/value/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_2/attention/self/Reshape_2
🟢 Final fusable group 19:
   + MatMul : bert/encoder/layer_2/attention/self/value/MatMul
   + Add : bert/encoder/layer_2/attention/self/value/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_2/attention/self/query/MatMul
   āœ… Fusable: Add : bert/encoder/layer_2/attention/self/query/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_2/attention/self/Reshape
🟢 Final fusable group 20:
   + MatMul : bert/encoder/layer_2/attention/self/query/MatMul
   + Add : bert/encoder/layer_2/attention/self/query/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_2/attention/self/key/MatMul
   āœ… Fusable: Add : bert/encoder/layer_2/attention/self/key/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_2/attention/self/Reshape_1
🟢 Final fusable group 21:
   + MatMul : bert/encoder/layer_2/attention/self/key/MatMul
   + Add : bert/encoder/layer_2/attention/self/key/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_2/attention/self/MatMul
   āœ… Fusable: Mul : bert/encoder/layer_2/attention/self/Mul
   āœ… Fusable: Add : bert/encoder/layer_2/attention/self/add
   ā›” Encountered non-elementwise: Softmax : bert/encoder/layer_2/attention/self/Softmax
🟢 Final fusable group 22:
   + MatMul : bert/encoder/layer_2/attention/self/MatMul
   + Mul : bert/encoder/layer_2/attention/self/Mul
   + Add : bert/encoder/layer_2/attention/self/add
šŸ”„ Starting from: MatMul : bert/encoder/layer_2/attention/self/MatMul_1
   ā›” Encountered non-elementwise: Transpose : bert/encoder/layer_2/attention/self/transpose_3
āš ļø No fusable chain found.
šŸ”„ Starting from: MatMul : bert/encoder/layer_2/attention/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_2/attention/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_2/attention/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_2/attention/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 23:
   + MatMul : bert/encoder/layer_2/attention/output/dense/MatMul
   + Add : bert/encoder/layer_2/attention/output/dense/BiasAdd
   + Add : bert/encoder/layer_2/attention/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_2/attention/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/Rsqrt__339
   āœ… Fusable: Mul : bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_2/attention/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 24:
   + Conv : bert/encoder/layer_2/attention/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/Rsqrt__339
   + Mul : bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_2/intermediate/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_2/intermediate/dense/BiasAdd
šŸ” Exploring multi-output node: Add : bert/encoder/layer_2/intermediate/dense/BiasAdd
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_2/intermediate/dense/mul_3
   āœ… Fusable: Mul : bert/encoder/layer_2/intermediate/dense/mul_3
   ā›” Encountered non-elementwise: MatMul : bert/encoder/layer_2/output/dense/MatMul
🟢 Final fusable group 25:
   + MatMul : bert/encoder/layer_2/intermediate/dense/MatMul
   + Add : bert/encoder/layer_2/intermediate/dense/BiasAdd
   + Mul : bert/encoder/layer_2/intermediate/dense/mul_2
   + Add : bert/encoder/layer_2/intermediate/dense/add_1
   + Tanh : bert/encoder/layer_2/intermediate/dense/Tanh
   + Mul : bert/encoder/layer_2/intermediate/dense/mul_1
   + Mul : bert/encoder/layer_2/intermediate/dense/mul
   + Add : bert/encoder/layer_2/intermediate/dense/add
   + Pow : bert/encoder/layer_2/intermediate/dense/Pow
   + Mul : bert/encoder/layer_2/intermediate/dense/mul_3
šŸ”„ Starting from: MatMul : bert/encoder/layer_2/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_2/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_2/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_2/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_2/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 26:
   + MatMul : bert/encoder/layer_2/output/dense/MatMul
   + Add : bert/encoder/layer_2/output/dense/BiasAdd
   + Add : bert/encoder/layer_2/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_2/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_2/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_2/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_2/output/LayerNorm/batchnorm/Rsqrt__343
   āœ… Fusable: Mul : bert/encoder/layer_2/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_2/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 27:
   + Conv : bert/encoder/layer_2/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_2/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_2/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_2/output/LayerNorm/batchnorm/Rsqrt__343
   + Mul : bert/encoder/layer_2/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_3/attention/self/value/MatMul
   āœ… Fusable: Add : bert/encoder/layer_3/attention/self/value/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_3/attention/self/Reshape_2
🟢 Final fusable group 28:
   + MatMul : bert/encoder/layer_3/attention/self/value/MatMul
   + Add : bert/encoder/layer_3/attention/self/value/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_3/attention/self/query/MatMul
   āœ… Fusable: Add : bert/encoder/layer_3/attention/self/query/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_3/attention/self/Reshape
🟢 Final fusable group 29:
   + MatMul : bert/encoder/layer_3/attention/self/query/MatMul
   + Add : bert/encoder/layer_3/attention/self/query/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_3/attention/self/key/MatMul
   āœ… Fusable: Add : bert/encoder/layer_3/attention/self/key/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_3/attention/self/Reshape_1
🟢 Final fusable group 30:
   + MatMul : bert/encoder/layer_3/attention/self/key/MatMul
   + Add : bert/encoder/layer_3/attention/self/key/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_3/attention/self/MatMul
   āœ… Fusable: Mul : bert/encoder/layer_3/attention/self/Mul
   āœ… Fusable: Add : bert/encoder/layer_3/attention/self/add
   ā›” Encountered non-elementwise: Softmax : bert/encoder/layer_3/attention/self/Softmax
🟢 Final fusable group 31:
   + MatMul : bert/encoder/layer_3/attention/self/MatMul
   + Mul : bert/encoder/layer_3/attention/self/Mul
   + Add : bert/encoder/layer_3/attention/self/add
šŸ”„ Starting from: MatMul : bert/encoder/layer_3/attention/self/MatMul_1
   ā›” Encountered non-elementwise: Transpose : bert/encoder/layer_3/attention/self/transpose_3
āš ļø No fusable chain found.
šŸ”„ Starting from: MatMul : bert/encoder/layer_3/attention/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_3/attention/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_3/attention/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_3/attention/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 32:
   + MatMul : bert/encoder/layer_3/attention/output/dense/MatMul
   + Add : bert/encoder/layer_3/attention/output/dense/BiasAdd
   + Add : bert/encoder/layer_3/attention/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_3/attention/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/Rsqrt__353
   āœ… Fusable: Mul : bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_3/attention/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 33:
   + Conv : bert/encoder/layer_3/attention/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/Rsqrt__353
   + Mul : bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_3/intermediate/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_3/intermediate/dense/BiasAdd
šŸ” Exploring multi-output node: Add : bert/encoder/layer_3/intermediate/dense/BiasAdd
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_3/intermediate/dense/mul_3
   āœ… Fusable: Mul : bert/encoder/layer_3/intermediate/dense/mul_3
   ā›” Encountered non-elementwise: MatMul : bert/encoder/layer_3/output/dense/MatMul
🟢 Final fusable group 34:
   + MatMul : bert/encoder/layer_3/intermediate/dense/MatMul
   + Add : bert/encoder/layer_3/intermediate/dense/BiasAdd
   + Mul : bert/encoder/layer_3/intermediate/dense/mul_2
   + Add : bert/encoder/layer_3/intermediate/dense/add_1
   + Tanh : bert/encoder/layer_3/intermediate/dense/Tanh
   + Mul : bert/encoder/layer_3/intermediate/dense/mul_1
   + Mul : bert/encoder/layer_3/intermediate/dense/mul
   + Add : bert/encoder/layer_3/intermediate/dense/add
   + Pow : bert/encoder/layer_3/intermediate/dense/Pow
   + Mul : bert/encoder/layer_3/intermediate/dense/mul_3
šŸ”„ Starting from: MatMul : bert/encoder/layer_3/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_3/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_3/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_3/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_3/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 35:
   + MatMul : bert/encoder/layer_3/output/dense/MatMul
   + Add : bert/encoder/layer_3/output/dense/BiasAdd
   + Add : bert/encoder/layer_3/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_3/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_3/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_3/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_3/output/LayerNorm/batchnorm/Rsqrt__357
   āœ… Fusable: Mul : bert/encoder/layer_3/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_3/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 36:
   + Conv : bert/encoder/layer_3/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_3/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_3/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_3/output/LayerNorm/batchnorm/Rsqrt__357
   + Mul : bert/encoder/layer_3/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_4/attention/self/value/MatMul
   āœ… Fusable: Add : bert/encoder/layer_4/attention/self/value/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_4/attention/self/Reshape_2
🟢 Final fusable group 37:
   + MatMul : bert/encoder/layer_4/attention/self/value/MatMul
   + Add : bert/encoder/layer_4/attention/self/value/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_4/attention/self/query/MatMul
   āœ… Fusable: Add : bert/encoder/layer_4/attention/self/query/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_4/attention/self/Reshape
🟢 Final fusable group 38:
   + MatMul : bert/encoder/layer_4/attention/self/query/MatMul
   + Add : bert/encoder/layer_4/attention/self/query/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_4/attention/self/key/MatMul
   āœ… Fusable: Add : bert/encoder/layer_4/attention/self/key/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_4/attention/self/Reshape_1
🟢 Final fusable group 39:
   + MatMul : bert/encoder/layer_4/attention/self/key/MatMul
   + Add : bert/encoder/layer_4/attention/self/key/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_4/attention/self/MatMul
   āœ… Fusable: Mul : bert/encoder/layer_4/attention/self/Mul
   āœ… Fusable: Add : bert/encoder/layer_4/attention/self/add
   ā›” Encountered non-elementwise: Softmax : bert/encoder/layer_4/attention/self/Softmax
🟢 Final fusable group 40:
   + MatMul : bert/encoder/layer_4/attention/self/MatMul
   + Mul : bert/encoder/layer_4/attention/self/Mul
   + Add : bert/encoder/layer_4/attention/self/add
šŸ”„ Starting from: MatMul : bert/encoder/layer_4/attention/self/MatMul_1
   ā›” Encountered non-elementwise: Transpose : bert/encoder/layer_4/attention/self/transpose_3
āš ļø No fusable chain found.
šŸ”„ Starting from: MatMul : bert/encoder/layer_4/attention/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_4/attention/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_4/attention/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_4/attention/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 41:
   + MatMul : bert/encoder/layer_4/attention/output/dense/MatMul
   + Add : bert/encoder/layer_4/attention/output/dense/BiasAdd
   + Add : bert/encoder/layer_4/attention/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_4/attention/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/Rsqrt__367
   āœ… Fusable: Mul : bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_4/attention/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 42:
   + Conv : bert/encoder/layer_4/attention/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/Rsqrt__367
   + Mul : bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_4/intermediate/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_4/intermediate/dense/BiasAdd
šŸ” Exploring multi-output node: Add : bert/encoder/layer_4/intermediate/dense/BiasAdd
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_4/intermediate/dense/mul_3
   āœ… Fusable: Mul : bert/encoder/layer_4/intermediate/dense/mul_3
   ā›” Encountered non-elementwise: MatMul : bert/encoder/layer_4/output/dense/MatMul
🟢 Final fusable group 43:
   + MatMul : bert/encoder/layer_4/intermediate/dense/MatMul
   + Add : bert/encoder/layer_4/intermediate/dense/BiasAdd
   + Mul : bert/encoder/layer_4/intermediate/dense/mul_2
   + Add : bert/encoder/layer_4/intermediate/dense/add_1
   + Tanh : bert/encoder/layer_4/intermediate/dense/Tanh
   + Mul : bert/encoder/layer_4/intermediate/dense/mul_1
   + Mul : bert/encoder/layer_4/intermediate/dense/mul
   + Add : bert/encoder/layer_4/intermediate/dense/add
   + Pow : bert/encoder/layer_4/intermediate/dense/Pow
   + Mul : bert/encoder/layer_4/intermediate/dense/mul_3
šŸ”„ Starting from: MatMul : bert/encoder/layer_4/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_4/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_4/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_4/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_4/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 44:
   + MatMul : bert/encoder/layer_4/output/dense/MatMul
   + Add : bert/encoder/layer_4/output/dense/BiasAdd
   + Add : bert/encoder/layer_4/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_4/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_4/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_4/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_4/output/LayerNorm/batchnorm/Rsqrt__371
   āœ… Fusable: Mul : bert/encoder/layer_4/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_4/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 45:
   + Conv : bert/encoder/layer_4/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_4/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_4/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_4/output/LayerNorm/batchnorm/Rsqrt__371
   + Mul : bert/encoder/layer_4/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_5/attention/self/value/MatMul
   āœ… Fusable: Add : bert/encoder/layer_5/attention/self/value/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_5/attention/self/Reshape_2
🟢 Final fusable group 46:
   + MatMul : bert/encoder/layer_5/attention/self/value/MatMul
   + Add : bert/encoder/layer_5/attention/self/value/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_5/attention/self/query/MatMul
   āœ… Fusable: Add : bert/encoder/layer_5/attention/self/query/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_5/attention/self/Reshape
🟢 Final fusable group 47:
   + MatMul : bert/encoder/layer_5/attention/self/query/MatMul
   + Add : bert/encoder/layer_5/attention/self/query/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_5/attention/self/key/MatMul
   āœ… Fusable: Add : bert/encoder/layer_5/attention/self/key/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_5/attention/self/Reshape_1
🟢 Final fusable group 48:
   + MatMul : bert/encoder/layer_5/attention/self/key/MatMul
   + Add : bert/encoder/layer_5/attention/self/key/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_5/attention/self/MatMul
   āœ… Fusable: Mul : bert/encoder/layer_5/attention/self/Mul
   āœ… Fusable: Add : bert/encoder/layer_5/attention/self/add
   ā›” Encountered non-elementwise: Softmax : bert/encoder/layer_5/attention/self/Softmax
🟢 Final fusable group 49:
   + MatMul : bert/encoder/layer_5/attention/self/MatMul
   + Mul : bert/encoder/layer_5/attention/self/Mul
   + Add : bert/encoder/layer_5/attention/self/add
šŸ”„ Starting from: MatMul : bert/encoder/layer_5/attention/self/MatMul_1
   ā›” Encountered non-elementwise: Transpose : bert/encoder/layer_5/attention/self/transpose_3
āš ļø No fusable chain found.
šŸ”„ Starting from: MatMul : bert/encoder/layer_5/attention/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_5/attention/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_5/attention/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_5/attention/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 50:
   + MatMul : bert/encoder/layer_5/attention/output/dense/MatMul
   + Add : bert/encoder/layer_5/attention/output/dense/BiasAdd
   + Add : bert/encoder/layer_5/attention/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_5/attention/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/Rsqrt__381
   āœ… Fusable: Mul : bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_5/attention/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 51:
   + Conv : bert/encoder/layer_5/attention/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/Rsqrt__381
   + Mul : bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_5/intermediate/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_5/intermediate/dense/BiasAdd
šŸ” Exploring multi-output node: Add : bert/encoder/layer_5/intermediate/dense/BiasAdd
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_5/intermediate/dense/mul_3
   āœ… Fusable: Mul : bert/encoder/layer_5/intermediate/dense/mul_3
   ā›” Encountered non-elementwise: MatMul : bert/encoder/layer_5/output/dense/MatMul
🟢 Final fusable group 52:
   + MatMul : bert/encoder/layer_5/intermediate/dense/MatMul
   + Add : bert/encoder/layer_5/intermediate/dense/BiasAdd
   + Mul : bert/encoder/layer_5/intermediate/dense/mul_2
   + Add : bert/encoder/layer_5/intermediate/dense/add_1
   + Tanh : bert/encoder/layer_5/intermediate/dense/Tanh
   + Mul : bert/encoder/layer_5/intermediate/dense/mul_1
   + Mul : bert/encoder/layer_5/intermediate/dense/mul
   + Add : bert/encoder/layer_5/intermediate/dense/add
   + Pow : bert/encoder/layer_5/intermediate/dense/Pow
   + Mul : bert/encoder/layer_5/intermediate/dense/mul_3
šŸ”„ Starting from: MatMul : bert/encoder/layer_5/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_5/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_5/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_5/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_5/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 53:
   + MatMul : bert/encoder/layer_5/output/dense/MatMul
   + Add : bert/encoder/layer_5/output/dense/BiasAdd
   + Add : bert/encoder/layer_5/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_5/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_5/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_5/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_5/output/LayerNorm/batchnorm/Rsqrt__385
   āœ… Fusable: Mul : bert/encoder/layer_5/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_5/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 54:
   + Conv : bert/encoder/layer_5/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_5/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_5/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_5/output/LayerNorm/batchnorm/Rsqrt__385
   + Mul : bert/encoder/layer_5/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_6/attention/self/value/MatMul
   āœ… Fusable: Add : bert/encoder/layer_6/attention/self/value/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_6/attention/self/Reshape_2
🟢 Final fusable group 55:
   + MatMul : bert/encoder/layer_6/attention/self/value/MatMul
   + Add : bert/encoder/layer_6/attention/self/value/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_6/attention/self/query/MatMul
   āœ… Fusable: Add : bert/encoder/layer_6/attention/self/query/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_6/attention/self/Reshape
🟢 Final fusable group 56:
   + MatMul : bert/encoder/layer_6/attention/self/query/MatMul
   + Add : bert/encoder/layer_6/attention/self/query/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_6/attention/self/key/MatMul
   āœ… Fusable: Add : bert/encoder/layer_6/attention/self/key/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_6/attention/self/Reshape_1
🟢 Final fusable group 57:
   + MatMul : bert/encoder/layer_6/attention/self/key/MatMul
   + Add : bert/encoder/layer_6/attention/self/key/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_6/attention/self/MatMul
   āœ… Fusable: Mul : bert/encoder/layer_6/attention/self/Mul
   āœ… Fusable: Add : bert/encoder/layer_6/attention/self/add
   ā›” Encountered non-elementwise: Softmax : bert/encoder/layer_6/attention/self/Softmax
🟢 Final fusable group 58:
   + MatMul : bert/encoder/layer_6/attention/self/MatMul
   + Mul : bert/encoder/layer_6/attention/self/Mul
   + Add : bert/encoder/layer_6/attention/self/add
šŸ”„ Starting from: MatMul : bert/encoder/layer_6/attention/self/MatMul_1
   ā›” Encountered non-elementwise: Transpose : bert/encoder/layer_6/attention/self/transpose_3
āš ļø No fusable chain found.
šŸ”„ Starting from: MatMul : bert/encoder/layer_6/attention/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_6/attention/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_6/attention/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_6/attention/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 59:
   + MatMul : bert/encoder/layer_6/attention/output/dense/MatMul
   + Add : bert/encoder/layer_6/attention/output/dense/BiasAdd
   + Add : bert/encoder/layer_6/attention/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_6/attention/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/Rsqrt__395
   āœ… Fusable: Mul : bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_6/attention/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 60:
   + Conv : bert/encoder/layer_6/attention/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/Rsqrt__395
   + Mul : bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_6/intermediate/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_6/intermediate/dense/BiasAdd
šŸ” Exploring multi-output node: Add : bert/encoder/layer_6/intermediate/dense/BiasAdd
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_6/intermediate/dense/mul_3
   āœ… Fusable: Mul : bert/encoder/layer_6/intermediate/dense/mul_3
   ā›” Encountered non-elementwise: MatMul : bert/encoder/layer_6/output/dense/MatMul
🟢 Final fusable group 61:
   + MatMul : bert/encoder/layer_6/intermediate/dense/MatMul
   + Add : bert/encoder/layer_6/intermediate/dense/BiasAdd
   + Mul : bert/encoder/layer_6/intermediate/dense/mul_2
   + Add : bert/encoder/layer_6/intermediate/dense/add_1
   + Tanh : bert/encoder/layer_6/intermediate/dense/Tanh
   + Mul : bert/encoder/layer_6/intermediate/dense/mul_1
   + Mul : bert/encoder/layer_6/intermediate/dense/mul
   + Add : bert/encoder/layer_6/intermediate/dense/add
   + Pow : bert/encoder/layer_6/intermediate/dense/Pow
   + Mul : bert/encoder/layer_6/intermediate/dense/mul_3
šŸ”„ Starting from: MatMul : bert/encoder/layer_6/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_6/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_6/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_6/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_6/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 62:
   + MatMul : bert/encoder/layer_6/output/dense/MatMul
   + Add : bert/encoder/layer_6/output/dense/BiasAdd
   + Add : bert/encoder/layer_6/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_6/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_6/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_6/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_6/output/LayerNorm/batchnorm/Rsqrt__399
   āœ… Fusable: Mul : bert/encoder/layer_6/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_6/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 63:
   + Conv : bert/encoder/layer_6/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_6/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_6/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_6/output/LayerNorm/batchnorm/Rsqrt__399
   + Mul : bert/encoder/layer_6/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_7/attention/self/value/MatMul
   āœ… Fusable: Add : bert/encoder/layer_7/attention/self/value/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_7/attention/self/Reshape_2
🟢 Final fusable group 64:
   + MatMul : bert/encoder/layer_7/attention/self/value/MatMul
   + Add : bert/encoder/layer_7/attention/self/value/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_7/attention/self/query/MatMul
   āœ… Fusable: Add : bert/encoder/layer_7/attention/self/query/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_7/attention/self/Reshape
🟢 Final fusable group 65:
   + MatMul : bert/encoder/layer_7/attention/self/query/MatMul
   + Add : bert/encoder/layer_7/attention/self/query/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_7/attention/self/key/MatMul
   āœ… Fusable: Add : bert/encoder/layer_7/attention/self/key/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_7/attention/self/Reshape_1
🟢 Final fusable group 66:
   + MatMul : bert/encoder/layer_7/attention/self/key/MatMul
   + Add : bert/encoder/layer_7/attention/self/key/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_7/attention/self/MatMul
   āœ… Fusable: Mul : bert/encoder/layer_7/attention/self/Mul
   āœ… Fusable: Add : bert/encoder/layer_7/attention/self/add
   ā›” Encountered non-elementwise: Softmax : bert/encoder/layer_7/attention/self/Softmax
🟢 Final fusable group 67:
   + MatMul : bert/encoder/layer_7/attention/self/MatMul
   + Mul : bert/encoder/layer_7/attention/self/Mul
   + Add : bert/encoder/layer_7/attention/self/add
šŸ”„ Starting from: MatMul : bert/encoder/layer_7/attention/self/MatMul_1
   ā›” Encountered non-elementwise: Transpose : bert/encoder/layer_7/attention/self/transpose_3
āš ļø No fusable chain found.
šŸ”„ Starting from: MatMul : bert/encoder/layer_7/attention/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_7/attention/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_7/attention/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_7/attention/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 68:
   + MatMul : bert/encoder/layer_7/attention/output/dense/MatMul
   + Add : bert/encoder/layer_7/attention/output/dense/BiasAdd
   + Add : bert/encoder/layer_7/attention/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_7/attention/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/Rsqrt__409
   āœ… Fusable: Mul : bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_7/attention/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 69:
   + Conv : bert/encoder/layer_7/attention/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/Rsqrt__409
   + Mul : bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_7/intermediate/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_7/intermediate/dense/BiasAdd
šŸ” Exploring multi-output node: Add : bert/encoder/layer_7/intermediate/dense/BiasAdd
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_7/intermediate/dense/mul_3
   āœ… Fusable: Mul : bert/encoder/layer_7/intermediate/dense/mul_3
   ā›” Encountered non-elementwise: MatMul : bert/encoder/layer_7/output/dense/MatMul
🟢 Final fusable group 70:
   + MatMul : bert/encoder/layer_7/intermediate/dense/MatMul
   + Add : bert/encoder/layer_7/intermediate/dense/BiasAdd
   + Mul : bert/encoder/layer_7/intermediate/dense/mul_2
   + Add : bert/encoder/layer_7/intermediate/dense/add_1
   + Tanh : bert/encoder/layer_7/intermediate/dense/Tanh
   + Mul : bert/encoder/layer_7/intermediate/dense/mul_1
   + Mul : bert/encoder/layer_7/intermediate/dense/mul
   + Add : bert/encoder/layer_7/intermediate/dense/add
   + Pow : bert/encoder/layer_7/intermediate/dense/Pow
   + Mul : bert/encoder/layer_7/intermediate/dense/mul_3
šŸ”„ Starting from: MatMul : bert/encoder/layer_7/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_7/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_7/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_7/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_7/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 71:
   + MatMul : bert/encoder/layer_7/output/dense/MatMul
   + Add : bert/encoder/layer_7/output/dense/BiasAdd
   + Add : bert/encoder/layer_7/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_7/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_7/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_7/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_7/output/LayerNorm/batchnorm/Rsqrt__413
   āœ… Fusable: Mul : bert/encoder/layer_7/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_7/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 72:
   + Conv : bert/encoder/layer_7/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_7/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_7/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_7/output/LayerNorm/batchnorm/Rsqrt__413
   + Mul : bert/encoder/layer_7/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_8/attention/self/value/MatMul
   āœ… Fusable: Add : bert/encoder/layer_8/attention/self/value/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_8/attention/self/Reshape_2
🟢 Final fusable group 73:
   + MatMul : bert/encoder/layer_8/attention/self/value/MatMul
   + Add : bert/encoder/layer_8/attention/self/value/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_8/attention/self/query/MatMul
   āœ… Fusable: Add : bert/encoder/layer_8/attention/self/query/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_8/attention/self/Reshape
🟢 Final fusable group 74:
   + MatMul : bert/encoder/layer_8/attention/self/query/MatMul
   + Add : bert/encoder/layer_8/attention/self/query/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_8/attention/self/key/MatMul
   āœ… Fusable: Add : bert/encoder/layer_8/attention/self/key/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_8/attention/self/Reshape_1
🟢 Final fusable group 75:
   + MatMul : bert/encoder/layer_8/attention/self/key/MatMul
   + Add : bert/encoder/layer_8/attention/self/key/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_8/attention/self/MatMul
   āœ… Fusable: Mul : bert/encoder/layer_8/attention/self/Mul
   āœ… Fusable: Add : bert/encoder/layer_8/attention/self/add
   ā›” Encountered non-elementwise: Softmax : bert/encoder/layer_8/attention/self/Softmax
🟢 Final fusable group 76:
   + MatMul : bert/encoder/layer_8/attention/self/MatMul
   + Mul : bert/encoder/layer_8/attention/self/Mul
   + Add : bert/encoder/layer_8/attention/self/add
šŸ”„ Starting from: MatMul : bert/encoder/layer_8/attention/self/MatMul_1
   ā›” Encountered non-elementwise: Transpose : bert/encoder/layer_8/attention/self/transpose_3
āš ļø No fusable chain found.
šŸ”„ Starting from: MatMul : bert/encoder/layer_8/attention/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_8/attention/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_8/attention/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_8/attention/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 77:
   + MatMul : bert/encoder/layer_8/attention/output/dense/MatMul
   + Add : bert/encoder/layer_8/attention/output/dense/BiasAdd
   + Add : bert/encoder/layer_8/attention/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_8/attention/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/Rsqrt__423
   āœ… Fusable: Mul : bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_8/attention/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 78:
   + Conv : bert/encoder/layer_8/attention/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/Rsqrt__423
   + Mul : bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_8/intermediate/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_8/intermediate/dense/BiasAdd
šŸ” Exploring multi-output node: Add : bert/encoder/layer_8/intermediate/dense/BiasAdd
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_8/intermediate/dense/mul_3
   āœ… Fusable: Mul : bert/encoder/layer_8/intermediate/dense/mul_3
   ā›” Encountered non-elementwise: MatMul : bert/encoder/layer_8/output/dense/MatMul
🟢 Final fusable group 79:
   + MatMul : bert/encoder/layer_8/intermediate/dense/MatMul
   + Add : bert/encoder/layer_8/intermediate/dense/BiasAdd
   + Mul : bert/encoder/layer_8/intermediate/dense/mul_2
   + Add : bert/encoder/layer_8/intermediate/dense/add_1
   + Tanh : bert/encoder/layer_8/intermediate/dense/Tanh
   + Mul : bert/encoder/layer_8/intermediate/dense/mul_1
   + Mul : bert/encoder/layer_8/intermediate/dense/mul
   + Add : bert/encoder/layer_8/intermediate/dense/add
   + Pow : bert/encoder/layer_8/intermediate/dense/Pow
   + Mul : bert/encoder/layer_8/intermediate/dense/mul_3
šŸ”„ Starting from: MatMul : bert/encoder/layer_8/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_8/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_8/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_8/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_8/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 80:
   + MatMul : bert/encoder/layer_8/output/dense/MatMul
   + Add : bert/encoder/layer_8/output/dense/BiasAdd
   + Add : bert/encoder/layer_8/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_8/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_8/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_8/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_8/output/LayerNorm/batchnorm/Rsqrt__427
   āœ… Fusable: Mul : bert/encoder/layer_8/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_8/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 81:
   + Conv : bert/encoder/layer_8/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_8/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_8/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_8/output/LayerNorm/batchnorm/Rsqrt__427
   + Mul : bert/encoder/layer_8/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_9/attention/self/value/MatMul
   āœ… Fusable: Add : bert/encoder/layer_9/attention/self/value/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_9/attention/self/Reshape_2
🟢 Final fusable group 82:
   + MatMul : bert/encoder/layer_9/attention/self/value/MatMul
   + Add : bert/encoder/layer_9/attention/self/value/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_9/attention/self/query/MatMul
   āœ… Fusable: Add : bert/encoder/layer_9/attention/self/query/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_9/attention/self/Reshape
🟢 Final fusable group 83:
   + MatMul : bert/encoder/layer_9/attention/self/query/MatMul
   + Add : bert/encoder/layer_9/attention/self/query/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_9/attention/self/key/MatMul
   āœ… Fusable: Add : bert/encoder/layer_9/attention/self/key/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_9/attention/self/Reshape_1
🟢 Final fusable group 84:
   + MatMul : bert/encoder/layer_9/attention/self/key/MatMul
   + Add : bert/encoder/layer_9/attention/self/key/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_9/attention/self/MatMul
   āœ… Fusable: Mul : bert/encoder/layer_9/attention/self/Mul
   āœ… Fusable: Add : bert/encoder/layer_9/attention/self/add
   ā›” Encountered non-elementwise: Softmax : bert/encoder/layer_9/attention/self/Softmax
🟢 Final fusable group 85:
   + MatMul : bert/encoder/layer_9/attention/self/MatMul
   + Mul : bert/encoder/layer_9/attention/self/Mul
   + Add : bert/encoder/layer_9/attention/self/add
šŸ”„ Starting from: MatMul : bert/encoder/layer_9/attention/self/MatMul_1
   ā›” Encountered non-elementwise: Transpose : bert/encoder/layer_9/attention/self/transpose_3
āš ļø No fusable chain found.
šŸ”„ Starting from: MatMul : bert/encoder/layer_9/attention/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_9/attention/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_9/attention/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_9/attention/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 86:
   + MatMul : bert/encoder/layer_9/attention/output/dense/MatMul
   + Add : bert/encoder/layer_9/attention/output/dense/BiasAdd
   + Add : bert/encoder/layer_9/attention/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_9/attention/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/Rsqrt__437
   āœ… Fusable: Mul : bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_9/attention/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 87:
   + Conv : bert/encoder/layer_9/attention/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/Rsqrt__437
   + Mul : bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_9/intermediate/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_9/intermediate/dense/BiasAdd
šŸ” Exploring multi-output node: Add : bert/encoder/layer_9/intermediate/dense/BiasAdd
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_9/intermediate/dense/mul_3
   āœ… Fusable: Mul : bert/encoder/layer_9/intermediate/dense/mul_3
   ā›” Encountered non-elementwise: MatMul : bert/encoder/layer_9/output/dense/MatMul
🟢 Final fusable group 88:
   + MatMul : bert/encoder/layer_9/intermediate/dense/MatMul
   + Add : bert/encoder/layer_9/intermediate/dense/BiasAdd
   + Mul : bert/encoder/layer_9/intermediate/dense/mul_2
   + Add : bert/encoder/layer_9/intermediate/dense/add_1
   + Tanh : bert/encoder/layer_9/intermediate/dense/Tanh
   + Mul : bert/encoder/layer_9/intermediate/dense/mul_1
   + Mul : bert/encoder/layer_9/intermediate/dense/mul
   + Add : bert/encoder/layer_9/intermediate/dense/add
   + Pow : bert/encoder/layer_9/intermediate/dense/Pow
   + Mul : bert/encoder/layer_9/intermediate/dense/mul_3
šŸ”„ Starting from: MatMul : bert/encoder/layer_9/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_9/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_9/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_9/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_9/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 89:
   + MatMul : bert/encoder/layer_9/output/dense/MatMul
   + Add : bert/encoder/layer_9/output/dense/BiasAdd
   + Add : bert/encoder/layer_9/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_9/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_9/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_9/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_9/output/LayerNorm/batchnorm/Rsqrt__441
   āœ… Fusable: Mul : bert/encoder/layer_9/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_9/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 90:
   + Conv : bert/encoder/layer_9/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_9/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_9/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_9/output/LayerNorm/batchnorm/Rsqrt__441
   + Mul : bert/encoder/layer_9/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_10/attention/self/value/MatMul
   āœ… Fusable: Add : bert/encoder/layer_10/attention/self/value/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_10/attention/self/Reshape_2
🟢 Final fusable group 91:
   + MatMul : bert/encoder/layer_10/attention/self/value/MatMul
   + Add : bert/encoder/layer_10/attention/self/value/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_10/attention/self/query/MatMul
   āœ… Fusable: Add : bert/encoder/layer_10/attention/self/query/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_10/attention/self/Reshape
🟢 Final fusable group 92:
   + MatMul : bert/encoder/layer_10/attention/self/query/MatMul
   + Add : bert/encoder/layer_10/attention/self/query/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_10/attention/self/key/MatMul
   āœ… Fusable: Add : bert/encoder/layer_10/attention/self/key/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_10/attention/self/Reshape_1
🟢 Final fusable group 93:
   + MatMul : bert/encoder/layer_10/attention/self/key/MatMul
   + Add : bert/encoder/layer_10/attention/self/key/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_10/attention/self/MatMul
   āœ… Fusable: Mul : bert/encoder/layer_10/attention/self/Mul
   āœ… Fusable: Add : bert/encoder/layer_10/attention/self/add
   ā›” Encountered non-elementwise: Softmax : bert/encoder/layer_10/attention/self/Softmax
🟢 Final fusable group 94:
   + MatMul : bert/encoder/layer_10/attention/self/MatMul
   + Mul : bert/encoder/layer_10/attention/self/Mul
   + Add : bert/encoder/layer_10/attention/self/add
šŸ”„ Starting from: MatMul : bert/encoder/layer_10/attention/self/MatMul_1
   ā›” Encountered non-elementwise: Transpose : bert/encoder/layer_10/attention/self/transpose_3
āš ļø No fusable chain found.
šŸ”„ Starting from: MatMul : bert/encoder/layer_10/attention/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_10/attention/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_10/attention/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_10/attention/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 95:
   + MatMul : bert/encoder/layer_10/attention/output/dense/MatMul
   + Add : bert/encoder/layer_10/attention/output/dense/BiasAdd
   + Add : bert/encoder/layer_10/attention/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_10/attention/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/Rsqrt__451
   āœ… Fusable: Mul : bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_10/attention/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 96:
   + Conv : bert/encoder/layer_10/attention/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/Rsqrt__451
   + Mul : bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_10/intermediate/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_10/intermediate/dense/BiasAdd
šŸ” Exploring multi-output node: Add : bert/encoder/layer_10/intermediate/dense/BiasAdd
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_10/intermediate/dense/mul_3
   āœ… Fusable: Mul : bert/encoder/layer_10/intermediate/dense/mul_3
   ā›” Encountered non-elementwise: MatMul : bert/encoder/layer_10/output/dense/MatMul
🟢 Final fusable group 97:
   + MatMul : bert/encoder/layer_10/intermediate/dense/MatMul
   + Add : bert/encoder/layer_10/intermediate/dense/BiasAdd
   + Mul : bert/encoder/layer_10/intermediate/dense/mul_2
   + Add : bert/encoder/layer_10/intermediate/dense/add_1
   + Tanh : bert/encoder/layer_10/intermediate/dense/Tanh
   + Mul : bert/encoder/layer_10/intermediate/dense/mul_1
   + Mul : bert/encoder/layer_10/intermediate/dense/mul
   + Add : bert/encoder/layer_10/intermediate/dense/add
   + Pow : bert/encoder/layer_10/intermediate/dense/Pow
   + Mul : bert/encoder/layer_10/intermediate/dense/mul_3
šŸ”„ Starting from: MatMul : bert/encoder/layer_10/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_10/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_10/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_10/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_10/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 98:
   + MatMul : bert/encoder/layer_10/output/dense/MatMul
   + Add : bert/encoder/layer_10/output/dense/BiasAdd
   + Add : bert/encoder/layer_10/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_10/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_10/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_10/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_10/output/LayerNorm/batchnorm/Rsqrt__455
   āœ… Fusable: Mul : bert/encoder/layer_10/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_10/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 99:
   + Conv : bert/encoder/layer_10/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_10/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_10/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_10/output/LayerNorm/batchnorm/Rsqrt__455
   + Mul : bert/encoder/layer_10/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_11/attention/self/value/MatMul
   āœ… Fusable: Add : bert/encoder/layer_11/attention/self/value/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_11/attention/self/Reshape_2
🟢 Final fusable group 100:
   + MatMul : bert/encoder/layer_11/attention/self/value/MatMul
   + Add : bert/encoder/layer_11/attention/self/value/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_11/attention/self/query/MatMul
   āœ… Fusable: Add : bert/encoder/layer_11/attention/self/query/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_11/attention/self/Reshape
🟢 Final fusable group 101:
   + MatMul : bert/encoder/layer_11/attention/self/query/MatMul
   + Add : bert/encoder/layer_11/attention/self/query/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_11/attention/self/key/MatMul
   āœ… Fusable: Add : bert/encoder/layer_11/attention/self/key/BiasAdd
   ā›” Encountered non-elementwise: Reshape : bert/encoder/layer_11/attention/self/Reshape_1
🟢 Final fusable group 102:
   + MatMul : bert/encoder/layer_11/attention/self/key/MatMul
   + Add : bert/encoder/layer_11/attention/self/key/BiasAdd
šŸ”„ Starting from: MatMul : bert/encoder/layer_11/attention/self/MatMul
   āœ… Fusable: Mul : bert/encoder/layer_11/attention/self/Mul
   āœ… Fusable: Add : bert/encoder/layer_11/attention/self/add
   ā›” Encountered non-elementwise: Softmax : bert/encoder/layer_11/attention/self/Softmax
🟢 Final fusable group 103:
   + MatMul : bert/encoder/layer_11/attention/self/MatMul
   + Mul : bert/encoder/layer_11/attention/self/Mul
   + Add : bert/encoder/layer_11/attention/self/add
šŸ”„ Starting from: MatMul : bert/encoder/layer_11/attention/self/MatMul_1
   ā›” Encountered non-elementwise: Transpose : bert/encoder/layer_11/attention/self/transpose_3
āš ļø No fusable chain found.
šŸ”„ Starting from: MatMul : bert/encoder/layer_11/attention/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_11/attention/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_11/attention/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_11/attention/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 104:
   + MatMul : bert/encoder/layer_11/attention/output/dense/MatMul
   + Add : bert/encoder/layer_11/attention/output/dense/BiasAdd
   + Add : bert/encoder/layer_11/attention/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_11/attention/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/Rsqrt__465
   āœ… Fusable: Mul : bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_11/attention/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 105:
   + Conv : bert/encoder/layer_11/attention/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/Rsqrt__465
   + Mul : bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : bert/encoder/layer_11/intermediate/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_11/intermediate/dense/BiasAdd
šŸ” Exploring multi-output node: Add : bert/encoder/layer_11/intermediate/dense/BiasAdd
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_11/intermediate/dense/mul_3
   āœ… Fusable: Mul : bert/encoder/layer_11/intermediate/dense/mul_3
   ā›” Encountered non-elementwise: MatMul : bert/encoder/layer_11/output/dense/MatMul
🟢 Final fusable group 106:
   + MatMul : bert/encoder/layer_11/intermediate/dense/MatMul
   + Add : bert/encoder/layer_11/intermediate/dense/BiasAdd
   + Mul : bert/encoder/layer_11/intermediate/dense/mul_2
   + Add : bert/encoder/layer_11/intermediate/dense/add_1
   + Tanh : bert/encoder/layer_11/intermediate/dense/Tanh
   + Mul : bert/encoder/layer_11/intermediate/dense/mul_1
   + Mul : bert/encoder/layer_11/intermediate/dense/mul
   + Add : bert/encoder/layer_11/intermediate/dense/add
   + Pow : bert/encoder/layer_11/intermediate/dense/Pow
   + Mul : bert/encoder/layer_11/intermediate/dense/mul_3
šŸ”„ Starting from: MatMul : bert/encoder/layer_11/output/dense/MatMul
   āœ… Fusable: Add : bert/encoder/layer_11/output/dense/BiasAdd
   āœ… Fusable: Add : bert/encoder/layer_11/output/add
šŸ” Exploring multi-output node: Add : bert/encoder/layer_11/output/add
āœ… Found earliest merge point:    → Mul : bert/encoder/layer_11/output/LayerNorm/batchnorm/mul_1
   ā›” Non-elementwise in path before merge point.
🟢 Final fusable group 107:
   + MatMul : bert/encoder/layer_11/output/dense/MatMul
   + Add : bert/encoder/layer_11/output/dense/BiasAdd
   + Add : bert/encoder/layer_11/output/add
šŸ”„ Starting from: Conv : bert/encoder/layer_11/output/LayerNorm/moments/variance/conv
   āœ… Fusable: Add : bert/encoder/layer_11/output/LayerNorm/batchnorm/add
   āœ… Fusable: Sqrt : bert/encoder/layer_11/output/LayerNorm/batchnorm/Rsqrt
   āœ… Fusable: Reciprocal : bert/encoder/layer_11/output/LayerNorm/batchnorm/Rsqrt__469
   āœ… Fusable: Mul : bert/encoder/layer_11/output/LayerNorm/batchnorm/mul
   ā›” Encountered non-elementwise: Squeeze : bert/encoder/layer_11/output/LayerNorm/moments/variance/squeeze
🟢 Final fusable group 108:
   + Conv : bert/encoder/layer_11/output/LayerNorm/moments/variance/conv
   + Add : bert/encoder/layer_11/output/LayerNorm/batchnorm/add
   + Sqrt : bert/encoder/layer_11/output/LayerNorm/batchnorm/Rsqrt
   + Reciprocal : bert/encoder/layer_11/output/LayerNorm/batchnorm/Rsqrt__469
   + Mul : bert/encoder/layer_11/output/LayerNorm/batchnorm/mul
šŸ”„ Starting from: MatMul : MatMul
   āœ… Fusable: Add : BiasAdd
   ā›” Encountered non-elementwise: Reshape : Reshape_1
🟢 Final fusable group 109:
   + MatMul : MatMul
   + Add : BiasAdd
Editor is loading...
Leave a Comment