Untitled
unknown
c_cpp
24 days ago
75 kB
9
Indexable
š„ Starting from: MatMul : bert/embeddings/MatMul ā Encountered non-elementwise: Reshape : bert/embeddings/Reshape_3 ā ļø No fusable chain found. š„ Starting from: Conv : bert/embeddings/LayerNorm/moments/variance/conv ā Fusable: Add : bert/embeddings/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/embeddings/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/embeddings/LayerNorm/batchnorm/Rsqrt__74 ā Fusable: Mul : bert/embeddings/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/embeddings/LayerNorm/moments/variance/squeeze š¢ Final fusable group 0: + Conv : bert/embeddings/LayerNorm/moments/variance/conv + Add : bert/embeddings/LayerNorm/batchnorm/add + Sqrt : bert/embeddings/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/embeddings/LayerNorm/batchnorm/Rsqrt__74 + Mul : bert/embeddings/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_0/attention/self/value/MatMul ā Fusable: Add : bert/encoder/layer_0/attention/self/value/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_0/attention/self/Reshape_2 š¢ Final fusable group 1: + MatMul : bert/encoder/layer_0/attention/self/value/MatMul + Add : bert/encoder/layer_0/attention/self/value/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_0/attention/self/query/MatMul ā Fusable: Add : bert/encoder/layer_0/attention/self/query/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_0/attention/self/Reshape š¢ Final fusable group 2: + MatMul : bert/encoder/layer_0/attention/self/query/MatMul + Add : bert/encoder/layer_0/attention/self/query/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_0/attention/self/key/MatMul ā Fusable: Add : bert/encoder/layer_0/attention/self/key/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_0/attention/self/Reshape_1 š¢ Final fusable group 3: + MatMul : bert/encoder/layer_0/attention/self/key/MatMul + Add : bert/encoder/layer_0/attention/self/key/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_0/attention/self/MatMul ā Fusable: Mul : bert/encoder/layer_0/attention/self/Mul ā Fusable: Add : bert/encoder/layer_0/attention/self/add ā Encountered non-elementwise: Softmax : bert/encoder/layer_0/attention/self/Softmax š¢ Final fusable group 4: + MatMul : bert/encoder/layer_0/attention/self/MatMul + Mul : bert/encoder/layer_0/attention/self/Mul + Add : bert/encoder/layer_0/attention/self/add š„ Starting from: MatMul : bert/encoder/layer_0/attention/self/MatMul_1 ā Encountered non-elementwise: Transpose : bert/encoder/layer_0/attention/self/transpose_3 ā ļø No fusable chain found. š„ Starting from: MatMul : bert/encoder/layer_0/attention/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_0/attention/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_0/attention/output/add š Exploring multi-output node: Add : bert/encoder/layer_0/attention/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 5: + MatMul : bert/encoder/layer_0/attention/output/dense/MatMul + Add : bert/encoder/layer_0/attention/output/dense/BiasAdd + Add : bert/encoder/layer_0/attention/output/add š„ Starting from: Conv : bert/encoder/layer_0/attention/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/Rsqrt__311 ā Fusable: Mul : bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_0/attention/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 6: + Conv : bert/encoder/layer_0/attention/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/Rsqrt__311 + Mul : bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_0/intermediate/dense/MatMul ā Fusable: Add : bert/encoder/layer_0/intermediate/dense/BiasAdd š Exploring multi-output node: Add : bert/encoder/layer_0/intermediate/dense/BiasAdd ā Found earliest merge point: ā Mul : bert/encoder/layer_0/intermediate/dense/mul_3 ā Fusable: Mul : bert/encoder/layer_0/intermediate/dense/mul_3 ā Encountered non-elementwise: MatMul : bert/encoder/layer_0/output/dense/MatMul š¢ Final fusable group 7: + MatMul : bert/encoder/layer_0/intermediate/dense/MatMul + Add : bert/encoder/layer_0/intermediate/dense/BiasAdd + Mul : bert/encoder/layer_0/intermediate/dense/mul_2 + Add : bert/encoder/layer_0/intermediate/dense/add_1 + Tanh : bert/encoder/layer_0/intermediate/dense/Tanh + Mul : bert/encoder/layer_0/intermediate/dense/mul_1 + Mul : bert/encoder/layer_0/intermediate/dense/mul + Add : bert/encoder/layer_0/intermediate/dense/add + Pow : bert/encoder/layer_0/intermediate/dense/Pow + Mul : bert/encoder/layer_0/intermediate/dense/mul_3 š„ Starting from: MatMul : bert/encoder/layer_0/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_0/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_0/output/add š Exploring multi-output node: Add : bert/encoder/layer_0/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_0/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 8: + MatMul : bert/encoder/layer_0/output/dense/MatMul + Add : bert/encoder/layer_0/output/dense/BiasAdd + Add : bert/encoder/layer_0/output/add š„ Starting from: Conv : bert/encoder/layer_0/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_0/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_0/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_0/output/LayerNorm/batchnorm/Rsqrt__315 ā Fusable: Mul : bert/encoder/layer_0/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_0/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 9: + Conv : bert/encoder/layer_0/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_0/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_0/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_0/output/LayerNorm/batchnorm/Rsqrt__315 + Mul : bert/encoder/layer_0/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_1/attention/self/value/MatMul ā Fusable: Add : bert/encoder/layer_1/attention/self/value/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_1/attention/self/Reshape_2 š¢ Final fusable group 10: + MatMul : bert/encoder/layer_1/attention/self/value/MatMul + Add : bert/encoder/layer_1/attention/self/value/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_1/attention/self/query/MatMul ā Fusable: Add : bert/encoder/layer_1/attention/self/query/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_1/attention/self/Reshape š¢ Final fusable group 11: + MatMul : bert/encoder/layer_1/attention/self/query/MatMul + Add : bert/encoder/layer_1/attention/self/query/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_1/attention/self/key/MatMul ā Fusable: Add : bert/encoder/layer_1/attention/self/key/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_1/attention/self/Reshape_1 š¢ Final fusable group 12: + MatMul : bert/encoder/layer_1/attention/self/key/MatMul + Add : bert/encoder/layer_1/attention/self/key/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_1/attention/self/MatMul ā Fusable: Mul : bert/encoder/layer_1/attention/self/Mul ā Fusable: Add : bert/encoder/layer_1/attention/self/add ā Encountered non-elementwise: Softmax : bert/encoder/layer_1/attention/self/Softmax š¢ Final fusable group 13: + MatMul : bert/encoder/layer_1/attention/self/MatMul + Mul : bert/encoder/layer_1/attention/self/Mul + Add : bert/encoder/layer_1/attention/self/add š„ Starting from: MatMul : bert/encoder/layer_1/attention/self/MatMul_1 ā Encountered non-elementwise: Transpose : bert/encoder/layer_1/attention/self/transpose_3 ā ļø No fusable chain found. š„ Starting from: MatMul : bert/encoder/layer_1/attention/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_1/attention/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_1/attention/output/add š Exploring multi-output node: Add : bert/encoder/layer_1/attention/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 14: + MatMul : bert/encoder/layer_1/attention/output/dense/MatMul + Add : bert/encoder/layer_1/attention/output/dense/BiasAdd + Add : bert/encoder/layer_1/attention/output/add š„ Starting from: Conv : bert/encoder/layer_1/attention/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/Rsqrt__325 ā Fusable: Mul : bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_1/attention/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 15: + Conv : bert/encoder/layer_1/attention/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/Rsqrt__325 + Mul : bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_1/intermediate/dense/MatMul ā Fusable: Add : bert/encoder/layer_1/intermediate/dense/BiasAdd š Exploring multi-output node: Add : bert/encoder/layer_1/intermediate/dense/BiasAdd ā Found earliest merge point: ā Mul : bert/encoder/layer_1/intermediate/dense/mul_3 ā Fusable: Mul : bert/encoder/layer_1/intermediate/dense/mul_3 ā Encountered non-elementwise: MatMul : bert/encoder/layer_1/output/dense/MatMul š¢ Final fusable group 16: + MatMul : bert/encoder/layer_1/intermediate/dense/MatMul + Add : bert/encoder/layer_1/intermediate/dense/BiasAdd + Mul : bert/encoder/layer_1/intermediate/dense/mul_2 + Add : bert/encoder/layer_1/intermediate/dense/add_1 + Tanh : bert/encoder/layer_1/intermediate/dense/Tanh + Mul : bert/encoder/layer_1/intermediate/dense/mul_1 + Mul : bert/encoder/layer_1/intermediate/dense/mul + Add : bert/encoder/layer_1/intermediate/dense/add + Pow : bert/encoder/layer_1/intermediate/dense/Pow + Mul : bert/encoder/layer_1/intermediate/dense/mul_3 š„ Starting from: MatMul : bert/encoder/layer_1/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_1/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_1/output/add š Exploring multi-output node: Add : bert/encoder/layer_1/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_1/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 17: + MatMul : bert/encoder/layer_1/output/dense/MatMul + Add : bert/encoder/layer_1/output/dense/BiasAdd + Add : bert/encoder/layer_1/output/add š„ Starting from: Conv : bert/encoder/layer_1/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_1/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_1/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_1/output/LayerNorm/batchnorm/Rsqrt__329 ā Fusable: Mul : bert/encoder/layer_1/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_1/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 18: + Conv : bert/encoder/layer_1/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_1/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_1/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_1/output/LayerNorm/batchnorm/Rsqrt__329 + Mul : bert/encoder/layer_1/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_2/attention/self/value/MatMul ā Fusable: Add : bert/encoder/layer_2/attention/self/value/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_2/attention/self/Reshape_2 š¢ Final fusable group 19: + MatMul : bert/encoder/layer_2/attention/self/value/MatMul + Add : bert/encoder/layer_2/attention/self/value/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_2/attention/self/query/MatMul ā Fusable: Add : bert/encoder/layer_2/attention/self/query/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_2/attention/self/Reshape š¢ Final fusable group 20: + MatMul : bert/encoder/layer_2/attention/self/query/MatMul + Add : bert/encoder/layer_2/attention/self/query/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_2/attention/self/key/MatMul ā Fusable: Add : bert/encoder/layer_2/attention/self/key/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_2/attention/self/Reshape_1 š¢ Final fusable group 21: + MatMul : bert/encoder/layer_2/attention/self/key/MatMul + Add : bert/encoder/layer_2/attention/self/key/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_2/attention/self/MatMul ā Fusable: Mul : bert/encoder/layer_2/attention/self/Mul ā Fusable: Add : bert/encoder/layer_2/attention/self/add ā Encountered non-elementwise: Softmax : bert/encoder/layer_2/attention/self/Softmax š¢ Final fusable group 22: + MatMul : bert/encoder/layer_2/attention/self/MatMul + Mul : bert/encoder/layer_2/attention/self/Mul + Add : bert/encoder/layer_2/attention/self/add š„ Starting from: MatMul : bert/encoder/layer_2/attention/self/MatMul_1 ā Encountered non-elementwise: Transpose : bert/encoder/layer_2/attention/self/transpose_3 ā ļø No fusable chain found. š„ Starting from: MatMul : bert/encoder/layer_2/attention/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_2/attention/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_2/attention/output/add š Exploring multi-output node: Add : bert/encoder/layer_2/attention/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 23: + MatMul : bert/encoder/layer_2/attention/output/dense/MatMul + Add : bert/encoder/layer_2/attention/output/dense/BiasAdd + Add : bert/encoder/layer_2/attention/output/add š„ Starting from: Conv : bert/encoder/layer_2/attention/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/Rsqrt__339 ā Fusable: Mul : bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_2/attention/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 24: + Conv : bert/encoder/layer_2/attention/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/Rsqrt__339 + Mul : bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_2/intermediate/dense/MatMul ā Fusable: Add : bert/encoder/layer_2/intermediate/dense/BiasAdd š Exploring multi-output node: Add : bert/encoder/layer_2/intermediate/dense/BiasAdd ā Found earliest merge point: ā Mul : bert/encoder/layer_2/intermediate/dense/mul_3 ā Fusable: Mul : bert/encoder/layer_2/intermediate/dense/mul_3 ā Encountered non-elementwise: MatMul : bert/encoder/layer_2/output/dense/MatMul š¢ Final fusable group 25: + MatMul : bert/encoder/layer_2/intermediate/dense/MatMul + Add : bert/encoder/layer_2/intermediate/dense/BiasAdd + Mul : bert/encoder/layer_2/intermediate/dense/mul_2 + Add : bert/encoder/layer_2/intermediate/dense/add_1 + Tanh : bert/encoder/layer_2/intermediate/dense/Tanh + Mul : bert/encoder/layer_2/intermediate/dense/mul_1 + Mul : bert/encoder/layer_2/intermediate/dense/mul + Add : bert/encoder/layer_2/intermediate/dense/add + Pow : bert/encoder/layer_2/intermediate/dense/Pow + Mul : bert/encoder/layer_2/intermediate/dense/mul_3 š„ Starting from: MatMul : bert/encoder/layer_2/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_2/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_2/output/add š Exploring multi-output node: Add : bert/encoder/layer_2/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_2/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 26: + MatMul : bert/encoder/layer_2/output/dense/MatMul + Add : bert/encoder/layer_2/output/dense/BiasAdd + Add : bert/encoder/layer_2/output/add š„ Starting from: Conv : bert/encoder/layer_2/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_2/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_2/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_2/output/LayerNorm/batchnorm/Rsqrt__343 ā Fusable: Mul : bert/encoder/layer_2/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_2/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 27: + Conv : bert/encoder/layer_2/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_2/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_2/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_2/output/LayerNorm/batchnorm/Rsqrt__343 + Mul : bert/encoder/layer_2/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_3/attention/self/value/MatMul ā Fusable: Add : bert/encoder/layer_3/attention/self/value/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_3/attention/self/Reshape_2 š¢ Final fusable group 28: + MatMul : bert/encoder/layer_3/attention/self/value/MatMul + Add : bert/encoder/layer_3/attention/self/value/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_3/attention/self/query/MatMul ā Fusable: Add : bert/encoder/layer_3/attention/self/query/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_3/attention/self/Reshape š¢ Final fusable group 29: + MatMul : bert/encoder/layer_3/attention/self/query/MatMul + Add : bert/encoder/layer_3/attention/self/query/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_3/attention/self/key/MatMul ā Fusable: Add : bert/encoder/layer_3/attention/self/key/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_3/attention/self/Reshape_1 š¢ Final fusable group 30: + MatMul : bert/encoder/layer_3/attention/self/key/MatMul + Add : bert/encoder/layer_3/attention/self/key/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_3/attention/self/MatMul ā Fusable: Mul : bert/encoder/layer_3/attention/self/Mul ā Fusable: Add : bert/encoder/layer_3/attention/self/add ā Encountered non-elementwise: Softmax : bert/encoder/layer_3/attention/self/Softmax š¢ Final fusable group 31: + MatMul : bert/encoder/layer_3/attention/self/MatMul + Mul : bert/encoder/layer_3/attention/self/Mul + Add : bert/encoder/layer_3/attention/self/add š„ Starting from: MatMul : bert/encoder/layer_3/attention/self/MatMul_1 ā Encountered non-elementwise: Transpose : bert/encoder/layer_3/attention/self/transpose_3 ā ļø No fusable chain found. š„ Starting from: MatMul : bert/encoder/layer_3/attention/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_3/attention/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_3/attention/output/add š Exploring multi-output node: Add : bert/encoder/layer_3/attention/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 32: + MatMul : bert/encoder/layer_3/attention/output/dense/MatMul + Add : bert/encoder/layer_3/attention/output/dense/BiasAdd + Add : bert/encoder/layer_3/attention/output/add š„ Starting from: Conv : bert/encoder/layer_3/attention/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/Rsqrt__353 ā Fusable: Mul : bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_3/attention/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 33: + Conv : bert/encoder/layer_3/attention/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/Rsqrt__353 + Mul : bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_3/intermediate/dense/MatMul ā Fusable: Add : bert/encoder/layer_3/intermediate/dense/BiasAdd š Exploring multi-output node: Add : bert/encoder/layer_3/intermediate/dense/BiasAdd ā Found earliest merge point: ā Mul : bert/encoder/layer_3/intermediate/dense/mul_3 ā Fusable: Mul : bert/encoder/layer_3/intermediate/dense/mul_3 ā Encountered non-elementwise: MatMul : bert/encoder/layer_3/output/dense/MatMul š¢ Final fusable group 34: + MatMul : bert/encoder/layer_3/intermediate/dense/MatMul + Add : bert/encoder/layer_3/intermediate/dense/BiasAdd + Mul : bert/encoder/layer_3/intermediate/dense/mul_2 + Add : bert/encoder/layer_3/intermediate/dense/add_1 + Tanh : bert/encoder/layer_3/intermediate/dense/Tanh + Mul : bert/encoder/layer_3/intermediate/dense/mul_1 + Mul : bert/encoder/layer_3/intermediate/dense/mul + Add : bert/encoder/layer_3/intermediate/dense/add + Pow : bert/encoder/layer_3/intermediate/dense/Pow + Mul : bert/encoder/layer_3/intermediate/dense/mul_3 š„ Starting from: MatMul : bert/encoder/layer_3/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_3/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_3/output/add š Exploring multi-output node: Add : bert/encoder/layer_3/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_3/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 35: + MatMul : bert/encoder/layer_3/output/dense/MatMul + Add : bert/encoder/layer_3/output/dense/BiasAdd + Add : bert/encoder/layer_3/output/add š„ Starting from: Conv : bert/encoder/layer_3/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_3/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_3/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_3/output/LayerNorm/batchnorm/Rsqrt__357 ā Fusable: Mul : bert/encoder/layer_3/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_3/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 36: + Conv : bert/encoder/layer_3/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_3/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_3/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_3/output/LayerNorm/batchnorm/Rsqrt__357 + Mul : bert/encoder/layer_3/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_4/attention/self/value/MatMul ā Fusable: Add : bert/encoder/layer_4/attention/self/value/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_4/attention/self/Reshape_2 š¢ Final fusable group 37: + MatMul : bert/encoder/layer_4/attention/self/value/MatMul + Add : bert/encoder/layer_4/attention/self/value/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_4/attention/self/query/MatMul ā Fusable: Add : bert/encoder/layer_4/attention/self/query/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_4/attention/self/Reshape š¢ Final fusable group 38: + MatMul : bert/encoder/layer_4/attention/self/query/MatMul + Add : bert/encoder/layer_4/attention/self/query/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_4/attention/self/key/MatMul ā Fusable: Add : bert/encoder/layer_4/attention/self/key/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_4/attention/self/Reshape_1 š¢ Final fusable group 39: + MatMul : bert/encoder/layer_4/attention/self/key/MatMul + Add : bert/encoder/layer_4/attention/self/key/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_4/attention/self/MatMul ā Fusable: Mul : bert/encoder/layer_4/attention/self/Mul ā Fusable: Add : bert/encoder/layer_4/attention/self/add ā Encountered non-elementwise: Softmax : bert/encoder/layer_4/attention/self/Softmax š¢ Final fusable group 40: + MatMul : bert/encoder/layer_4/attention/self/MatMul + Mul : bert/encoder/layer_4/attention/self/Mul + Add : bert/encoder/layer_4/attention/self/add š„ Starting from: MatMul : bert/encoder/layer_4/attention/self/MatMul_1 ā Encountered non-elementwise: Transpose : bert/encoder/layer_4/attention/self/transpose_3 ā ļø No fusable chain found. š„ Starting from: MatMul : bert/encoder/layer_4/attention/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_4/attention/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_4/attention/output/add š Exploring multi-output node: Add : bert/encoder/layer_4/attention/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 41: + MatMul : bert/encoder/layer_4/attention/output/dense/MatMul + Add : bert/encoder/layer_4/attention/output/dense/BiasAdd + Add : bert/encoder/layer_4/attention/output/add š„ Starting from: Conv : bert/encoder/layer_4/attention/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/Rsqrt__367 ā Fusable: Mul : bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_4/attention/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 42: + Conv : bert/encoder/layer_4/attention/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/Rsqrt__367 + Mul : bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_4/intermediate/dense/MatMul ā Fusable: Add : bert/encoder/layer_4/intermediate/dense/BiasAdd š Exploring multi-output node: Add : bert/encoder/layer_4/intermediate/dense/BiasAdd ā Found earliest merge point: ā Mul : bert/encoder/layer_4/intermediate/dense/mul_3 ā Fusable: Mul : bert/encoder/layer_4/intermediate/dense/mul_3 ā Encountered non-elementwise: MatMul : bert/encoder/layer_4/output/dense/MatMul š¢ Final fusable group 43: + MatMul : bert/encoder/layer_4/intermediate/dense/MatMul + Add : bert/encoder/layer_4/intermediate/dense/BiasAdd + Mul : bert/encoder/layer_4/intermediate/dense/mul_2 + Add : bert/encoder/layer_4/intermediate/dense/add_1 + Tanh : bert/encoder/layer_4/intermediate/dense/Tanh + Mul : bert/encoder/layer_4/intermediate/dense/mul_1 + Mul : bert/encoder/layer_4/intermediate/dense/mul + Add : bert/encoder/layer_4/intermediate/dense/add + Pow : bert/encoder/layer_4/intermediate/dense/Pow + Mul : bert/encoder/layer_4/intermediate/dense/mul_3 š„ Starting from: MatMul : bert/encoder/layer_4/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_4/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_4/output/add š Exploring multi-output node: Add : bert/encoder/layer_4/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_4/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 44: + MatMul : bert/encoder/layer_4/output/dense/MatMul + Add : bert/encoder/layer_4/output/dense/BiasAdd + Add : bert/encoder/layer_4/output/add š„ Starting from: Conv : bert/encoder/layer_4/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_4/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_4/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_4/output/LayerNorm/batchnorm/Rsqrt__371 ā Fusable: Mul : bert/encoder/layer_4/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_4/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 45: + Conv : bert/encoder/layer_4/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_4/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_4/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_4/output/LayerNorm/batchnorm/Rsqrt__371 + Mul : bert/encoder/layer_4/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_5/attention/self/value/MatMul ā Fusable: Add : bert/encoder/layer_5/attention/self/value/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_5/attention/self/Reshape_2 š¢ Final fusable group 46: + MatMul : bert/encoder/layer_5/attention/self/value/MatMul + Add : bert/encoder/layer_5/attention/self/value/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_5/attention/self/query/MatMul ā Fusable: Add : bert/encoder/layer_5/attention/self/query/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_5/attention/self/Reshape š¢ Final fusable group 47: + MatMul : bert/encoder/layer_5/attention/self/query/MatMul + Add : bert/encoder/layer_5/attention/self/query/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_5/attention/self/key/MatMul ā Fusable: Add : bert/encoder/layer_5/attention/self/key/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_5/attention/self/Reshape_1 š¢ Final fusable group 48: + MatMul : bert/encoder/layer_5/attention/self/key/MatMul + Add : bert/encoder/layer_5/attention/self/key/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_5/attention/self/MatMul ā Fusable: Mul : bert/encoder/layer_5/attention/self/Mul ā Fusable: Add : bert/encoder/layer_5/attention/self/add ā Encountered non-elementwise: Softmax : bert/encoder/layer_5/attention/self/Softmax š¢ Final fusable group 49: + MatMul : bert/encoder/layer_5/attention/self/MatMul + Mul : bert/encoder/layer_5/attention/self/Mul + Add : bert/encoder/layer_5/attention/self/add š„ Starting from: MatMul : bert/encoder/layer_5/attention/self/MatMul_1 ā Encountered non-elementwise: Transpose : bert/encoder/layer_5/attention/self/transpose_3 ā ļø No fusable chain found. š„ Starting from: MatMul : bert/encoder/layer_5/attention/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_5/attention/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_5/attention/output/add š Exploring multi-output node: Add : bert/encoder/layer_5/attention/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 50: + MatMul : bert/encoder/layer_5/attention/output/dense/MatMul + Add : bert/encoder/layer_5/attention/output/dense/BiasAdd + Add : bert/encoder/layer_5/attention/output/add š„ Starting from: Conv : bert/encoder/layer_5/attention/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/Rsqrt__381 ā Fusable: Mul : bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_5/attention/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 51: + Conv : bert/encoder/layer_5/attention/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/Rsqrt__381 + Mul : bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_5/intermediate/dense/MatMul ā Fusable: Add : bert/encoder/layer_5/intermediate/dense/BiasAdd š Exploring multi-output node: Add : bert/encoder/layer_5/intermediate/dense/BiasAdd ā Found earliest merge point: ā Mul : bert/encoder/layer_5/intermediate/dense/mul_3 ā Fusable: Mul : bert/encoder/layer_5/intermediate/dense/mul_3 ā Encountered non-elementwise: MatMul : bert/encoder/layer_5/output/dense/MatMul š¢ Final fusable group 52: + MatMul : bert/encoder/layer_5/intermediate/dense/MatMul + Add : bert/encoder/layer_5/intermediate/dense/BiasAdd + Mul : bert/encoder/layer_5/intermediate/dense/mul_2 + Add : bert/encoder/layer_5/intermediate/dense/add_1 + Tanh : bert/encoder/layer_5/intermediate/dense/Tanh + Mul : bert/encoder/layer_5/intermediate/dense/mul_1 + Mul : bert/encoder/layer_5/intermediate/dense/mul + Add : bert/encoder/layer_5/intermediate/dense/add + Pow : bert/encoder/layer_5/intermediate/dense/Pow + Mul : bert/encoder/layer_5/intermediate/dense/mul_3 š„ Starting from: MatMul : bert/encoder/layer_5/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_5/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_5/output/add š Exploring multi-output node: Add : bert/encoder/layer_5/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_5/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 53: + MatMul : bert/encoder/layer_5/output/dense/MatMul + Add : bert/encoder/layer_5/output/dense/BiasAdd + Add : bert/encoder/layer_5/output/add š„ Starting from: Conv : bert/encoder/layer_5/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_5/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_5/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_5/output/LayerNorm/batchnorm/Rsqrt__385 ā Fusable: Mul : bert/encoder/layer_5/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_5/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 54: + Conv : bert/encoder/layer_5/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_5/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_5/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_5/output/LayerNorm/batchnorm/Rsqrt__385 + Mul : bert/encoder/layer_5/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_6/attention/self/value/MatMul ā Fusable: Add : bert/encoder/layer_6/attention/self/value/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_6/attention/self/Reshape_2 š¢ Final fusable group 55: + MatMul : bert/encoder/layer_6/attention/self/value/MatMul + Add : bert/encoder/layer_6/attention/self/value/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_6/attention/self/query/MatMul ā Fusable: Add : bert/encoder/layer_6/attention/self/query/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_6/attention/self/Reshape š¢ Final fusable group 56: + MatMul : bert/encoder/layer_6/attention/self/query/MatMul + Add : bert/encoder/layer_6/attention/self/query/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_6/attention/self/key/MatMul ā Fusable: Add : bert/encoder/layer_6/attention/self/key/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_6/attention/self/Reshape_1 š¢ Final fusable group 57: + MatMul : bert/encoder/layer_6/attention/self/key/MatMul + Add : bert/encoder/layer_6/attention/self/key/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_6/attention/self/MatMul ā Fusable: Mul : bert/encoder/layer_6/attention/self/Mul ā Fusable: Add : bert/encoder/layer_6/attention/self/add ā Encountered non-elementwise: Softmax : bert/encoder/layer_6/attention/self/Softmax š¢ Final fusable group 58: + MatMul : bert/encoder/layer_6/attention/self/MatMul + Mul : bert/encoder/layer_6/attention/self/Mul + Add : bert/encoder/layer_6/attention/self/add š„ Starting from: MatMul : bert/encoder/layer_6/attention/self/MatMul_1 ā Encountered non-elementwise: Transpose : bert/encoder/layer_6/attention/self/transpose_3 ā ļø No fusable chain found. š„ Starting from: MatMul : bert/encoder/layer_6/attention/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_6/attention/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_6/attention/output/add š Exploring multi-output node: Add : bert/encoder/layer_6/attention/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 59: + MatMul : bert/encoder/layer_6/attention/output/dense/MatMul + Add : bert/encoder/layer_6/attention/output/dense/BiasAdd + Add : bert/encoder/layer_6/attention/output/add š„ Starting from: Conv : bert/encoder/layer_6/attention/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/Rsqrt__395 ā Fusable: Mul : bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_6/attention/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 60: + Conv : bert/encoder/layer_6/attention/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/Rsqrt__395 + Mul : bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_6/intermediate/dense/MatMul ā Fusable: Add : bert/encoder/layer_6/intermediate/dense/BiasAdd š Exploring multi-output node: Add : bert/encoder/layer_6/intermediate/dense/BiasAdd ā Found earliest merge point: ā Mul : bert/encoder/layer_6/intermediate/dense/mul_3 ā Fusable: Mul : bert/encoder/layer_6/intermediate/dense/mul_3 ā Encountered non-elementwise: MatMul : bert/encoder/layer_6/output/dense/MatMul š¢ Final fusable group 61: + MatMul : bert/encoder/layer_6/intermediate/dense/MatMul + Add : bert/encoder/layer_6/intermediate/dense/BiasAdd + Mul : bert/encoder/layer_6/intermediate/dense/mul_2 + Add : bert/encoder/layer_6/intermediate/dense/add_1 + Tanh : bert/encoder/layer_6/intermediate/dense/Tanh + Mul : bert/encoder/layer_6/intermediate/dense/mul_1 + Mul : bert/encoder/layer_6/intermediate/dense/mul + Add : bert/encoder/layer_6/intermediate/dense/add + Pow : bert/encoder/layer_6/intermediate/dense/Pow + Mul : bert/encoder/layer_6/intermediate/dense/mul_3 š„ Starting from: MatMul : bert/encoder/layer_6/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_6/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_6/output/add š Exploring multi-output node: Add : bert/encoder/layer_6/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_6/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 62: + MatMul : bert/encoder/layer_6/output/dense/MatMul + Add : bert/encoder/layer_6/output/dense/BiasAdd + Add : bert/encoder/layer_6/output/add š„ Starting from: Conv : bert/encoder/layer_6/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_6/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_6/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_6/output/LayerNorm/batchnorm/Rsqrt__399 ā Fusable: Mul : bert/encoder/layer_6/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_6/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 63: + Conv : bert/encoder/layer_6/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_6/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_6/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_6/output/LayerNorm/batchnorm/Rsqrt__399 + Mul : bert/encoder/layer_6/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_7/attention/self/value/MatMul ā Fusable: Add : bert/encoder/layer_7/attention/self/value/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_7/attention/self/Reshape_2 š¢ Final fusable group 64: + MatMul : bert/encoder/layer_7/attention/self/value/MatMul + Add : bert/encoder/layer_7/attention/self/value/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_7/attention/self/query/MatMul ā Fusable: Add : bert/encoder/layer_7/attention/self/query/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_7/attention/self/Reshape š¢ Final fusable group 65: + MatMul : bert/encoder/layer_7/attention/self/query/MatMul + Add : bert/encoder/layer_7/attention/self/query/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_7/attention/self/key/MatMul ā Fusable: Add : bert/encoder/layer_7/attention/self/key/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_7/attention/self/Reshape_1 š¢ Final fusable group 66: + MatMul : bert/encoder/layer_7/attention/self/key/MatMul + Add : bert/encoder/layer_7/attention/self/key/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_7/attention/self/MatMul ā Fusable: Mul : bert/encoder/layer_7/attention/self/Mul ā Fusable: Add : bert/encoder/layer_7/attention/self/add ā Encountered non-elementwise: Softmax : bert/encoder/layer_7/attention/self/Softmax š¢ Final fusable group 67: + MatMul : bert/encoder/layer_7/attention/self/MatMul + Mul : bert/encoder/layer_7/attention/self/Mul + Add : bert/encoder/layer_7/attention/self/add š„ Starting from: MatMul : bert/encoder/layer_7/attention/self/MatMul_1 ā Encountered non-elementwise: Transpose : bert/encoder/layer_7/attention/self/transpose_3 ā ļø No fusable chain found. š„ Starting from: MatMul : bert/encoder/layer_7/attention/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_7/attention/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_7/attention/output/add š Exploring multi-output node: Add : bert/encoder/layer_7/attention/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 68: + MatMul : bert/encoder/layer_7/attention/output/dense/MatMul + Add : bert/encoder/layer_7/attention/output/dense/BiasAdd + Add : bert/encoder/layer_7/attention/output/add š„ Starting from: Conv : bert/encoder/layer_7/attention/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/Rsqrt__409 ā Fusable: Mul : bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_7/attention/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 69: + Conv : bert/encoder/layer_7/attention/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/Rsqrt__409 + Mul : bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_7/intermediate/dense/MatMul ā Fusable: Add : bert/encoder/layer_7/intermediate/dense/BiasAdd š Exploring multi-output node: Add : bert/encoder/layer_7/intermediate/dense/BiasAdd ā Found earliest merge point: ā Mul : bert/encoder/layer_7/intermediate/dense/mul_3 ā Fusable: Mul : bert/encoder/layer_7/intermediate/dense/mul_3 ā Encountered non-elementwise: MatMul : bert/encoder/layer_7/output/dense/MatMul š¢ Final fusable group 70: + MatMul : bert/encoder/layer_7/intermediate/dense/MatMul + Add : bert/encoder/layer_7/intermediate/dense/BiasAdd + Mul : bert/encoder/layer_7/intermediate/dense/mul_2 + Add : bert/encoder/layer_7/intermediate/dense/add_1 + Tanh : bert/encoder/layer_7/intermediate/dense/Tanh + Mul : bert/encoder/layer_7/intermediate/dense/mul_1 + Mul : bert/encoder/layer_7/intermediate/dense/mul + Add : bert/encoder/layer_7/intermediate/dense/add + Pow : bert/encoder/layer_7/intermediate/dense/Pow + Mul : bert/encoder/layer_7/intermediate/dense/mul_3 š„ Starting from: MatMul : bert/encoder/layer_7/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_7/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_7/output/add š Exploring multi-output node: Add : bert/encoder/layer_7/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_7/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 71: + MatMul : bert/encoder/layer_7/output/dense/MatMul + Add : bert/encoder/layer_7/output/dense/BiasAdd + Add : bert/encoder/layer_7/output/add š„ Starting from: Conv : bert/encoder/layer_7/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_7/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_7/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_7/output/LayerNorm/batchnorm/Rsqrt__413 ā Fusable: Mul : bert/encoder/layer_7/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_7/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 72: + Conv : bert/encoder/layer_7/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_7/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_7/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_7/output/LayerNorm/batchnorm/Rsqrt__413 + Mul : bert/encoder/layer_7/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_8/attention/self/value/MatMul ā Fusable: Add : bert/encoder/layer_8/attention/self/value/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_8/attention/self/Reshape_2 š¢ Final fusable group 73: + MatMul : bert/encoder/layer_8/attention/self/value/MatMul + Add : bert/encoder/layer_8/attention/self/value/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_8/attention/self/query/MatMul ā Fusable: Add : bert/encoder/layer_8/attention/self/query/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_8/attention/self/Reshape š¢ Final fusable group 74: + MatMul : bert/encoder/layer_8/attention/self/query/MatMul + Add : bert/encoder/layer_8/attention/self/query/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_8/attention/self/key/MatMul ā Fusable: Add : bert/encoder/layer_8/attention/self/key/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_8/attention/self/Reshape_1 š¢ Final fusable group 75: + MatMul : bert/encoder/layer_8/attention/self/key/MatMul + Add : bert/encoder/layer_8/attention/self/key/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_8/attention/self/MatMul ā Fusable: Mul : bert/encoder/layer_8/attention/self/Mul ā Fusable: Add : bert/encoder/layer_8/attention/self/add ā Encountered non-elementwise: Softmax : bert/encoder/layer_8/attention/self/Softmax š¢ Final fusable group 76: + MatMul : bert/encoder/layer_8/attention/self/MatMul + Mul : bert/encoder/layer_8/attention/self/Mul + Add : bert/encoder/layer_8/attention/self/add š„ Starting from: MatMul : bert/encoder/layer_8/attention/self/MatMul_1 ā Encountered non-elementwise: Transpose : bert/encoder/layer_8/attention/self/transpose_3 ā ļø No fusable chain found. š„ Starting from: MatMul : bert/encoder/layer_8/attention/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_8/attention/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_8/attention/output/add š Exploring multi-output node: Add : bert/encoder/layer_8/attention/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 77: + MatMul : bert/encoder/layer_8/attention/output/dense/MatMul + Add : bert/encoder/layer_8/attention/output/dense/BiasAdd + Add : bert/encoder/layer_8/attention/output/add š„ Starting from: Conv : bert/encoder/layer_8/attention/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/Rsqrt__423 ā Fusable: Mul : bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_8/attention/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 78: + Conv : bert/encoder/layer_8/attention/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/Rsqrt__423 + Mul : bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_8/intermediate/dense/MatMul ā Fusable: Add : bert/encoder/layer_8/intermediate/dense/BiasAdd š Exploring multi-output node: Add : bert/encoder/layer_8/intermediate/dense/BiasAdd ā Found earliest merge point: ā Mul : bert/encoder/layer_8/intermediate/dense/mul_3 ā Fusable: Mul : bert/encoder/layer_8/intermediate/dense/mul_3 ā Encountered non-elementwise: MatMul : bert/encoder/layer_8/output/dense/MatMul š¢ Final fusable group 79: + MatMul : bert/encoder/layer_8/intermediate/dense/MatMul + Add : bert/encoder/layer_8/intermediate/dense/BiasAdd + Mul : bert/encoder/layer_8/intermediate/dense/mul_2 + Add : bert/encoder/layer_8/intermediate/dense/add_1 + Tanh : bert/encoder/layer_8/intermediate/dense/Tanh + Mul : bert/encoder/layer_8/intermediate/dense/mul_1 + Mul : bert/encoder/layer_8/intermediate/dense/mul + Add : bert/encoder/layer_8/intermediate/dense/add + Pow : bert/encoder/layer_8/intermediate/dense/Pow + Mul : bert/encoder/layer_8/intermediate/dense/mul_3 š„ Starting from: MatMul : bert/encoder/layer_8/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_8/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_8/output/add š Exploring multi-output node: Add : bert/encoder/layer_8/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_8/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 80: + MatMul : bert/encoder/layer_8/output/dense/MatMul + Add : bert/encoder/layer_8/output/dense/BiasAdd + Add : bert/encoder/layer_8/output/add š„ Starting from: Conv : bert/encoder/layer_8/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_8/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_8/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_8/output/LayerNorm/batchnorm/Rsqrt__427 ā Fusable: Mul : bert/encoder/layer_8/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_8/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 81: + Conv : bert/encoder/layer_8/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_8/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_8/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_8/output/LayerNorm/batchnorm/Rsqrt__427 + Mul : bert/encoder/layer_8/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_9/attention/self/value/MatMul ā Fusable: Add : bert/encoder/layer_9/attention/self/value/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_9/attention/self/Reshape_2 š¢ Final fusable group 82: + MatMul : bert/encoder/layer_9/attention/self/value/MatMul + Add : bert/encoder/layer_9/attention/self/value/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_9/attention/self/query/MatMul ā Fusable: Add : bert/encoder/layer_9/attention/self/query/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_9/attention/self/Reshape š¢ Final fusable group 83: + MatMul : bert/encoder/layer_9/attention/self/query/MatMul + Add : bert/encoder/layer_9/attention/self/query/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_9/attention/self/key/MatMul ā Fusable: Add : bert/encoder/layer_9/attention/self/key/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_9/attention/self/Reshape_1 š¢ Final fusable group 84: + MatMul : bert/encoder/layer_9/attention/self/key/MatMul + Add : bert/encoder/layer_9/attention/self/key/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_9/attention/self/MatMul ā Fusable: Mul : bert/encoder/layer_9/attention/self/Mul ā Fusable: Add : bert/encoder/layer_9/attention/self/add ā Encountered non-elementwise: Softmax : bert/encoder/layer_9/attention/self/Softmax š¢ Final fusable group 85: + MatMul : bert/encoder/layer_9/attention/self/MatMul + Mul : bert/encoder/layer_9/attention/self/Mul + Add : bert/encoder/layer_9/attention/self/add š„ Starting from: MatMul : bert/encoder/layer_9/attention/self/MatMul_1 ā Encountered non-elementwise: Transpose : bert/encoder/layer_9/attention/self/transpose_3 ā ļø No fusable chain found. š„ Starting from: MatMul : bert/encoder/layer_9/attention/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_9/attention/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_9/attention/output/add š Exploring multi-output node: Add : bert/encoder/layer_9/attention/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 86: + MatMul : bert/encoder/layer_9/attention/output/dense/MatMul + Add : bert/encoder/layer_9/attention/output/dense/BiasAdd + Add : bert/encoder/layer_9/attention/output/add š„ Starting from: Conv : bert/encoder/layer_9/attention/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/Rsqrt__437 ā Fusable: Mul : bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_9/attention/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 87: + Conv : bert/encoder/layer_9/attention/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/Rsqrt__437 + Mul : bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_9/intermediate/dense/MatMul ā Fusable: Add : bert/encoder/layer_9/intermediate/dense/BiasAdd š Exploring multi-output node: Add : bert/encoder/layer_9/intermediate/dense/BiasAdd ā Found earliest merge point: ā Mul : bert/encoder/layer_9/intermediate/dense/mul_3 ā Fusable: Mul : bert/encoder/layer_9/intermediate/dense/mul_3 ā Encountered non-elementwise: MatMul : bert/encoder/layer_9/output/dense/MatMul š¢ Final fusable group 88: + MatMul : bert/encoder/layer_9/intermediate/dense/MatMul + Add : bert/encoder/layer_9/intermediate/dense/BiasAdd + Mul : bert/encoder/layer_9/intermediate/dense/mul_2 + Add : bert/encoder/layer_9/intermediate/dense/add_1 + Tanh : bert/encoder/layer_9/intermediate/dense/Tanh + Mul : bert/encoder/layer_9/intermediate/dense/mul_1 + Mul : bert/encoder/layer_9/intermediate/dense/mul + Add : bert/encoder/layer_9/intermediate/dense/add + Pow : bert/encoder/layer_9/intermediate/dense/Pow + Mul : bert/encoder/layer_9/intermediate/dense/mul_3 š„ Starting from: MatMul : bert/encoder/layer_9/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_9/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_9/output/add š Exploring multi-output node: Add : bert/encoder/layer_9/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_9/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 89: + MatMul : bert/encoder/layer_9/output/dense/MatMul + Add : bert/encoder/layer_9/output/dense/BiasAdd + Add : bert/encoder/layer_9/output/add š„ Starting from: Conv : bert/encoder/layer_9/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_9/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_9/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_9/output/LayerNorm/batchnorm/Rsqrt__441 ā Fusable: Mul : bert/encoder/layer_9/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_9/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 90: + Conv : bert/encoder/layer_9/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_9/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_9/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_9/output/LayerNorm/batchnorm/Rsqrt__441 + Mul : bert/encoder/layer_9/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_10/attention/self/value/MatMul ā Fusable: Add : bert/encoder/layer_10/attention/self/value/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_10/attention/self/Reshape_2 š¢ Final fusable group 91: + MatMul : bert/encoder/layer_10/attention/self/value/MatMul + Add : bert/encoder/layer_10/attention/self/value/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_10/attention/self/query/MatMul ā Fusable: Add : bert/encoder/layer_10/attention/self/query/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_10/attention/self/Reshape š¢ Final fusable group 92: + MatMul : bert/encoder/layer_10/attention/self/query/MatMul + Add : bert/encoder/layer_10/attention/self/query/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_10/attention/self/key/MatMul ā Fusable: Add : bert/encoder/layer_10/attention/self/key/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_10/attention/self/Reshape_1 š¢ Final fusable group 93: + MatMul : bert/encoder/layer_10/attention/self/key/MatMul + Add : bert/encoder/layer_10/attention/self/key/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_10/attention/self/MatMul ā Fusable: Mul : bert/encoder/layer_10/attention/self/Mul ā Fusable: Add : bert/encoder/layer_10/attention/self/add ā Encountered non-elementwise: Softmax : bert/encoder/layer_10/attention/self/Softmax š¢ Final fusable group 94: + MatMul : bert/encoder/layer_10/attention/self/MatMul + Mul : bert/encoder/layer_10/attention/self/Mul + Add : bert/encoder/layer_10/attention/self/add š„ Starting from: MatMul : bert/encoder/layer_10/attention/self/MatMul_1 ā Encountered non-elementwise: Transpose : bert/encoder/layer_10/attention/self/transpose_3 ā ļø No fusable chain found. š„ Starting from: MatMul : bert/encoder/layer_10/attention/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_10/attention/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_10/attention/output/add š Exploring multi-output node: Add : bert/encoder/layer_10/attention/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 95: + MatMul : bert/encoder/layer_10/attention/output/dense/MatMul + Add : bert/encoder/layer_10/attention/output/dense/BiasAdd + Add : bert/encoder/layer_10/attention/output/add š„ Starting from: Conv : bert/encoder/layer_10/attention/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/Rsqrt__451 ā Fusable: Mul : bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_10/attention/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 96: + Conv : bert/encoder/layer_10/attention/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/Rsqrt__451 + Mul : bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_10/intermediate/dense/MatMul ā Fusable: Add : bert/encoder/layer_10/intermediate/dense/BiasAdd š Exploring multi-output node: Add : bert/encoder/layer_10/intermediate/dense/BiasAdd ā Found earliest merge point: ā Mul : bert/encoder/layer_10/intermediate/dense/mul_3 ā Fusable: Mul : bert/encoder/layer_10/intermediate/dense/mul_3 ā Encountered non-elementwise: MatMul : bert/encoder/layer_10/output/dense/MatMul š¢ Final fusable group 97: + MatMul : bert/encoder/layer_10/intermediate/dense/MatMul + Add : bert/encoder/layer_10/intermediate/dense/BiasAdd + Mul : bert/encoder/layer_10/intermediate/dense/mul_2 + Add : bert/encoder/layer_10/intermediate/dense/add_1 + Tanh : bert/encoder/layer_10/intermediate/dense/Tanh + Mul : bert/encoder/layer_10/intermediate/dense/mul_1 + Mul : bert/encoder/layer_10/intermediate/dense/mul + Add : bert/encoder/layer_10/intermediate/dense/add + Pow : bert/encoder/layer_10/intermediate/dense/Pow + Mul : bert/encoder/layer_10/intermediate/dense/mul_3 š„ Starting from: MatMul : bert/encoder/layer_10/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_10/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_10/output/add š Exploring multi-output node: Add : bert/encoder/layer_10/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_10/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 98: + MatMul : bert/encoder/layer_10/output/dense/MatMul + Add : bert/encoder/layer_10/output/dense/BiasAdd + Add : bert/encoder/layer_10/output/add š„ Starting from: Conv : bert/encoder/layer_10/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_10/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_10/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_10/output/LayerNorm/batchnorm/Rsqrt__455 ā Fusable: Mul : bert/encoder/layer_10/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_10/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 99: + Conv : bert/encoder/layer_10/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_10/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_10/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_10/output/LayerNorm/batchnorm/Rsqrt__455 + Mul : bert/encoder/layer_10/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_11/attention/self/value/MatMul ā Fusable: Add : bert/encoder/layer_11/attention/self/value/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_11/attention/self/Reshape_2 š¢ Final fusable group 100: + MatMul : bert/encoder/layer_11/attention/self/value/MatMul + Add : bert/encoder/layer_11/attention/self/value/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_11/attention/self/query/MatMul ā Fusable: Add : bert/encoder/layer_11/attention/self/query/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_11/attention/self/Reshape š¢ Final fusable group 101: + MatMul : bert/encoder/layer_11/attention/self/query/MatMul + Add : bert/encoder/layer_11/attention/self/query/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_11/attention/self/key/MatMul ā Fusable: Add : bert/encoder/layer_11/attention/self/key/BiasAdd ā Encountered non-elementwise: Reshape : bert/encoder/layer_11/attention/self/Reshape_1 š¢ Final fusable group 102: + MatMul : bert/encoder/layer_11/attention/self/key/MatMul + Add : bert/encoder/layer_11/attention/self/key/BiasAdd š„ Starting from: MatMul : bert/encoder/layer_11/attention/self/MatMul ā Fusable: Mul : bert/encoder/layer_11/attention/self/Mul ā Fusable: Add : bert/encoder/layer_11/attention/self/add ā Encountered non-elementwise: Softmax : bert/encoder/layer_11/attention/self/Softmax š¢ Final fusable group 103: + MatMul : bert/encoder/layer_11/attention/self/MatMul + Mul : bert/encoder/layer_11/attention/self/Mul + Add : bert/encoder/layer_11/attention/self/add š„ Starting from: MatMul : bert/encoder/layer_11/attention/self/MatMul_1 ā Encountered non-elementwise: Transpose : bert/encoder/layer_11/attention/self/transpose_3 ā ļø No fusable chain found. š„ Starting from: MatMul : bert/encoder/layer_11/attention/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_11/attention/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_11/attention/output/add š Exploring multi-output node: Add : bert/encoder/layer_11/attention/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 104: + MatMul : bert/encoder/layer_11/attention/output/dense/MatMul + Add : bert/encoder/layer_11/attention/output/dense/BiasAdd + Add : bert/encoder/layer_11/attention/output/add š„ Starting from: Conv : bert/encoder/layer_11/attention/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/Rsqrt__465 ā Fusable: Mul : bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_11/attention/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 105: + Conv : bert/encoder/layer_11/attention/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/Rsqrt__465 + Mul : bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : bert/encoder/layer_11/intermediate/dense/MatMul ā Fusable: Add : bert/encoder/layer_11/intermediate/dense/BiasAdd š Exploring multi-output node: Add : bert/encoder/layer_11/intermediate/dense/BiasAdd ā Found earliest merge point: ā Mul : bert/encoder/layer_11/intermediate/dense/mul_3 ā Fusable: Mul : bert/encoder/layer_11/intermediate/dense/mul_3 ā Encountered non-elementwise: MatMul : bert/encoder/layer_11/output/dense/MatMul š¢ Final fusable group 106: + MatMul : bert/encoder/layer_11/intermediate/dense/MatMul + Add : bert/encoder/layer_11/intermediate/dense/BiasAdd + Mul : bert/encoder/layer_11/intermediate/dense/mul_2 + Add : bert/encoder/layer_11/intermediate/dense/add_1 + Tanh : bert/encoder/layer_11/intermediate/dense/Tanh + Mul : bert/encoder/layer_11/intermediate/dense/mul_1 + Mul : bert/encoder/layer_11/intermediate/dense/mul + Add : bert/encoder/layer_11/intermediate/dense/add + Pow : bert/encoder/layer_11/intermediate/dense/Pow + Mul : bert/encoder/layer_11/intermediate/dense/mul_3 š„ Starting from: MatMul : bert/encoder/layer_11/output/dense/MatMul ā Fusable: Add : bert/encoder/layer_11/output/dense/BiasAdd ā Fusable: Add : bert/encoder/layer_11/output/add š Exploring multi-output node: Add : bert/encoder/layer_11/output/add ā Found earliest merge point: ā Mul : bert/encoder/layer_11/output/LayerNorm/batchnorm/mul_1 ā Non-elementwise in path before merge point. š¢ Final fusable group 107: + MatMul : bert/encoder/layer_11/output/dense/MatMul + Add : bert/encoder/layer_11/output/dense/BiasAdd + Add : bert/encoder/layer_11/output/add š„ Starting from: Conv : bert/encoder/layer_11/output/LayerNorm/moments/variance/conv ā Fusable: Add : bert/encoder/layer_11/output/LayerNorm/batchnorm/add ā Fusable: Sqrt : bert/encoder/layer_11/output/LayerNorm/batchnorm/Rsqrt ā Fusable: Reciprocal : bert/encoder/layer_11/output/LayerNorm/batchnorm/Rsqrt__469 ā Fusable: Mul : bert/encoder/layer_11/output/LayerNorm/batchnorm/mul ā Encountered non-elementwise: Squeeze : bert/encoder/layer_11/output/LayerNorm/moments/variance/squeeze š¢ Final fusable group 108: + Conv : bert/encoder/layer_11/output/LayerNorm/moments/variance/conv + Add : bert/encoder/layer_11/output/LayerNorm/batchnorm/add + Sqrt : bert/encoder/layer_11/output/LayerNorm/batchnorm/Rsqrt + Reciprocal : bert/encoder/layer_11/output/LayerNorm/batchnorm/Rsqrt__469 + Mul : bert/encoder/layer_11/output/LayerNorm/batchnorm/mul š„ Starting from: MatMul : MatMul ā Fusable: Add : BiasAdd ā Encountered non-elementwise: Reshape : Reshape_1 š¢ Final fusable group 109: + MatMul : MatMul + Add : BiasAdd
Editor is loading...
Leave a Comment