Untitled

 avatar
unknown
c_cpp
a month ago
79 kB
1
Indexable
Model Input Name: unique_ids_raw_output___9:0, Shape: [0]
Model Input Name: segment_ids:0, Shape: [0, 256]
Model Input Name: input_mask:0, Shape: [0, 256]
Model Input Name: input_ids:0, Shape: [0, 256]
Starting model execution...

Inputs Details:
Input Name: input_ids:0
Shape: (1, 256)
Data (first 10 values): [14558 12889 27202  3336 19210 22185 18020 28836 27440 30487]...
--------------------------------------------------
Input Name: segment_ids:0
Shape: (1, 256)
Data (first 10 values): [0 1 1 0 1 0 0 0 1 1]...
--------------------------------------------------
Input Name: input_mask:0
Shape: (1, 256)
Data (first 10 values): [0 0 1 0 0 0 0 1 1 0]...
--------------------------------------------------
Input Name: unique_ids_raw_output___9:0
Shape: (0,)
Data (first 10 values): []...
--------------------------------------------------
No Add node related to MatMul output: bert/embeddings/MatMul. Executing regular MatMul.
Fusing MatMul with Add for Node: bert/encoder/layer_0/attention/self/value/MatMul
Skipping already processed Add Node: bert/encoder/layer_0/attention/self/value/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_0/attention/self/query/MatMul
Skipping already processed Add Node: bert/encoder/layer_0/attention/self/query/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_0/attention/self/key/MatMul
Skipping already processed Add Node: bert/encoder/layer_0/attention/self/key/BiasAdd
No Add node related to MatMul output: bert/encoder/layer_0/attention/self/MatMul. Executing regular MatMul.
No Add node related to MatMul output: bert/encoder/layer_0/attention/self/MatMul_1. Executing regular MatMul.
Fusing MatMul with 2Add for Node: bert/encoder/layer_0/attention/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_0/attention/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_0/attention/output/add
Fusing MatMul with Add for Node: bert/encoder/layer_0/intermediate/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_0/intermediate/dense/BiasAdd
Fusing MatMul with 2Add for Node: bert/encoder/layer_0/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_0/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_0/output/add
Fusing MatMul with Add for Node: bert/encoder/layer_1/attention/self/value/MatMul
Skipping already processed Add Node: bert/encoder/layer_1/attention/self/value/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_1/attention/self/query/MatMul
Skipping already processed Add Node: bert/encoder/layer_1/attention/self/query/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_1/attention/self/key/MatMul
Skipping already processed Add Node: bert/encoder/layer_1/attention/self/key/BiasAdd
No Add node related to MatMul output: bert/encoder/layer_1/attention/self/MatMul. Executing regular MatMul.
No Add node related to MatMul output: bert/encoder/layer_1/attention/self/MatMul_1. Executing regular MatMul.
Fusing MatMul with 2Add for Node: bert/encoder/layer_1/attention/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_1/attention/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_1/attention/output/add
Fusing MatMul with Add for Node: bert/encoder/layer_1/intermediate/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_1/intermediate/dense/BiasAdd
Fusing MatMul with 2Add for Node: bert/encoder/layer_1/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_1/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_1/output/add
Fusing MatMul with Add for Node: bert/encoder/layer_2/attention/self/value/MatMul
Skipping already processed Add Node: bert/encoder/layer_2/attention/self/value/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_2/attention/self/query/MatMul
Skipping already processed Add Node: bert/encoder/layer_2/attention/self/query/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_2/attention/self/key/MatMul
Skipping already processed Add Node: bert/encoder/layer_2/attention/self/key/BiasAdd
No Add node related to MatMul output: bert/encoder/layer_2/attention/self/MatMul. Executing regular MatMul.
No Add node related to MatMul output: bert/encoder/layer_2/attention/self/MatMul_1. Executing regular MatMul.
Fusing MatMul with 2Add for Node: bert/encoder/layer_2/attention/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_2/attention/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_2/attention/output/add
Fusing MatMul with Add for Node: bert/encoder/layer_2/intermediate/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_2/intermediate/dense/BiasAdd
Fusing MatMul with 2Add for Node: bert/encoder/layer_2/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_2/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_2/output/add
Fusing MatMul with Add for Node: bert/encoder/layer_3/attention/self/value/MatMul
Skipping already processed Add Node: bert/encoder/layer_3/attention/self/value/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_3/attention/self/query/MatMul
Skipping already processed Add Node: bert/encoder/layer_3/attention/self/query/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_3/attention/self/key/MatMul
Skipping already processed Add Node: bert/encoder/layer_3/attention/self/key/BiasAdd
No Add node related to MatMul output: bert/encoder/layer_3/attention/self/MatMul. Executing regular MatMul.
No Add node related to MatMul output: bert/encoder/layer_3/attention/self/MatMul_1. Executing regular MatMul.
Fusing MatMul with 2Add for Node: bert/encoder/layer_3/attention/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_3/attention/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_3/attention/output/add
Fusing MatMul with Add for Node: bert/encoder/layer_3/intermediate/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_3/intermediate/dense/BiasAdd
Fusing MatMul with 2Add for Node: bert/encoder/layer_3/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_3/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_3/output/add
Fusing MatMul with Add for Node: bert/encoder/layer_4/attention/self/value/MatMul
Skipping already processed Add Node: bert/encoder/layer_4/attention/self/value/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_4/attention/self/query/MatMul
Skipping already processed Add Node: bert/encoder/layer_4/attention/self/query/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_4/attention/self/key/MatMul
Skipping already processed Add Node: bert/encoder/layer_4/attention/self/key/BiasAdd
No Add node related to MatMul output: bert/encoder/layer_4/attention/self/MatMul. Executing regular MatMul.
No Add node related to MatMul output: bert/encoder/layer_4/attention/self/MatMul_1. Executing regular MatMul.
Fusing MatMul with 2Add for Node: bert/encoder/layer_4/attention/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_4/attention/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_4/attention/output/add
Fusing MatMul with Add for Node: bert/encoder/layer_4/intermediate/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_4/intermediate/dense/BiasAdd
Fusing MatMul with 2Add for Node: bert/encoder/layer_4/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_4/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_4/output/add
Fusing MatMul with Add for Node: bert/encoder/layer_5/attention/self/value/MatMul
Skipping already processed Add Node: bert/encoder/layer_5/attention/self/value/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_5/attention/self/query/MatMul
Skipping already processed Add Node: bert/encoder/layer_5/attention/self/query/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_5/attention/self/key/MatMul
Skipping already processed Add Node: bert/encoder/layer_5/attention/self/key/BiasAdd
No Add node related to MatMul output: bert/encoder/layer_5/attention/self/MatMul. Executing regular MatMul.
No Add node related to MatMul output: bert/encoder/layer_5/attention/self/MatMul_1. Executing regular MatMul.
Fusing MatMul with 2Add for Node: bert/encoder/layer_5/attention/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_5/attention/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_5/attention/output/add
Fusing MatMul with Add for Node: bert/encoder/layer_5/intermediate/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_5/intermediate/dense/BiasAdd
Fusing MatMul with 2Add for Node: bert/encoder/layer_5/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_5/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_5/output/add
Fusing MatMul with Add for Node: bert/encoder/layer_6/attention/self/value/MatMul
Skipping already processed Add Node: bert/encoder/layer_6/attention/self/value/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_6/attention/self/query/MatMul
Skipping already processed Add Node: bert/encoder/layer_6/attention/self/query/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_6/attention/self/key/MatMul
Skipping already processed Add Node: bert/encoder/layer_6/attention/self/key/BiasAdd
No Add node related to MatMul output: bert/encoder/layer_6/attention/self/MatMul. Executing regular MatMul.
No Add node related to MatMul output: bert/encoder/layer_6/attention/self/MatMul_1. Executing regular MatMul.
Fusing MatMul with 2Add for Node: bert/encoder/layer_6/attention/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_6/attention/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_6/attention/output/add
Fusing MatMul with Add for Node: bert/encoder/layer_6/intermediate/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_6/intermediate/dense/BiasAdd
Fusing MatMul with 2Add for Node: bert/encoder/layer_6/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_6/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_6/output/add
Fusing MatMul with Add for Node: bert/encoder/layer_7/attention/self/value/MatMul
Skipping already processed Add Node: bert/encoder/layer_7/attention/self/value/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_7/attention/self/query/MatMul
Skipping already processed Add Node: bert/encoder/layer_7/attention/self/query/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_7/attention/self/key/MatMul
Skipping already processed Add Node: bert/encoder/layer_7/attention/self/key/BiasAdd
No Add node related to MatMul output: bert/encoder/layer_7/attention/self/MatMul. Executing regular MatMul.
No Add node related to MatMul output: bert/encoder/layer_7/attention/self/MatMul_1. Executing regular MatMul.
Fusing MatMul with 2Add for Node: bert/encoder/layer_7/attention/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_7/attention/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_7/attention/output/add
Fusing MatMul with Add for Node: bert/encoder/layer_7/intermediate/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_7/intermediate/dense/BiasAdd
Fusing MatMul with 2Add for Node: bert/encoder/layer_7/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_7/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_7/output/add
Fusing MatMul with Add for Node: bert/encoder/layer_8/attention/self/value/MatMul
Skipping already processed Add Node: bert/encoder/layer_8/attention/self/value/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_8/attention/self/query/MatMul
Skipping already processed Add Node: bert/encoder/layer_8/attention/self/query/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_8/attention/self/key/MatMul
Skipping already processed Add Node: bert/encoder/layer_8/attention/self/key/BiasAdd
No Add node related to MatMul output: bert/encoder/layer_8/attention/self/MatMul. Executing regular MatMul.
No Add node related to MatMul output: bert/encoder/layer_8/attention/self/MatMul_1. Executing regular MatMul.
Fusing MatMul with 2Add for Node: bert/encoder/layer_8/attention/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_8/attention/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_8/attention/output/add
Fusing MatMul with Add for Node: bert/encoder/layer_8/intermediate/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_8/intermediate/dense/BiasAdd
Fusing MatMul with 2Add for Node: bert/encoder/layer_8/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_8/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_8/output/add
Fusing MatMul with Add for Node: bert/encoder/layer_9/attention/self/value/MatMul
Skipping already processed Add Node: bert/encoder/layer_9/attention/self/value/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_9/attention/self/query/MatMul
Skipping already processed Add Node: bert/encoder/layer_9/attention/self/query/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_9/attention/self/key/MatMul
Skipping already processed Add Node: bert/encoder/layer_9/attention/self/key/BiasAdd
No Add node related to MatMul output: bert/encoder/layer_9/attention/self/MatMul. Executing regular MatMul.
No Add node related to MatMul output: bert/encoder/layer_9/attention/self/MatMul_1. Executing regular MatMul.
Fusing MatMul with 2Add for Node: bert/encoder/layer_9/attention/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_9/attention/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_9/attention/output/add
Fusing MatMul with Add for Node: bert/encoder/layer_9/intermediate/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_9/intermediate/dense/BiasAdd
Fusing MatMul with 2Add for Node: bert/encoder/layer_9/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_9/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_9/output/add
Fusing MatMul with Add for Node: bert/encoder/layer_10/attention/self/value/MatMul
Skipping already processed Add Node: bert/encoder/layer_10/attention/self/value/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_10/attention/self/query/MatMul
Skipping already processed Add Node: bert/encoder/layer_10/attention/self/query/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_10/attention/self/key/MatMul
Skipping already processed Add Node: bert/encoder/layer_10/attention/self/key/BiasAdd
No Add node related to MatMul output: bert/encoder/layer_10/attention/self/MatMul. Executing regular MatMul.
No Add node related to MatMul output: bert/encoder/layer_10/attention/self/MatMul_1. Executing regular MatMul.
Fusing MatMul with 2Add for Node: bert/encoder/layer_10/attention/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_10/attention/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_10/attention/output/add
Fusing MatMul with Add for Node: bert/encoder/layer_10/intermediate/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_10/intermediate/dense/BiasAdd
Fusing MatMul with 2Add for Node: bert/encoder/layer_10/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_10/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_10/output/add
Fusing MatMul with Add for Node: bert/encoder/layer_11/attention/self/value/MatMul
Skipping already processed Add Node: bert/encoder/layer_11/attention/self/value/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_11/attention/self/query/MatMul
Skipping already processed Add Node: bert/encoder/layer_11/attention/self/query/BiasAdd
Fusing MatMul with Add for Node: bert/encoder/layer_11/attention/self/key/MatMul
Skipping already processed Add Node: bert/encoder/layer_11/attention/self/key/BiasAdd
No Add node related to MatMul output: bert/encoder/layer_11/attention/self/MatMul. Executing regular MatMul.
No Add node related to MatMul output: bert/encoder/layer_11/attention/self/MatMul_1. Executing regular MatMul.
Fusing MatMul with 2Add for Node: bert/encoder/layer_11/attention/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_11/attention/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_11/attention/output/add
Fusing MatMul with Add for Node: bert/encoder/layer_11/intermediate/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_11/intermediate/dense/BiasAdd
Fusing MatMul with 2Add for Node: bert/encoder/layer_11/output/dense/MatMul
Skipping already processed Add Node: bert/encoder/layer_11/output/dense/BiasAdd
Skipping already processed Add Node: bert/encoder/layer_11/output/add
Fusing MatMul with Add for Node: MatMul
Skipping already processed Add Node: BiasAdd

Node Execution Times:
Node: unique_ids_graph_outputs_Identity__10, Execution Time: 0.000005 seconds
Node: bert/encoder/Shape, Execution Time: 0.000003 seconds
Node: bert/encoder/Shape__12, Execution Time: 0.000009 seconds
Node: bert/encoder/strided_slice, Execution Time: 0.000072 seconds
Node: bert/encoder/strided_slice__16, Execution Time: 0.000005 seconds
Node: bert/encoder/strided_slice__17, Execution Time: 0.000005 seconds
Node: bert/encoder/ones/packed_Unsqueeze__18, Execution Time: 0.000010 seconds
Node: bert/encoder/ones/packed_Concat__21, Execution Time: 0.000010 seconds
Node: bert/encoder/ones__22, Execution Time: 0.000004 seconds
Node: bert/encoder/ones, Execution Time: 0.000010 seconds
Node: bert/encoder/Reshape, Execution Time: 0.000006 seconds
Node: bert/encoder/Cast, Execution Time: 0.000004 seconds
Node: bert/encoder/mul, Execution Time: 0.028060 seconds
Node: bert/encoder/layer_9/attention/self/ExpandDims, Execution Time: 0.000035 seconds
Node: bert/encoder/layer_9/attention/self/sub, Execution Time: 0.006613 seconds
Node: bert/encoder/layer_9/attention/self/mul_1, Execution Time: 0.000350 seconds
Node: bert/embeddings/Reshape_2, Execution Time: 0.000030 seconds
Node: bert/embeddings/Reshape, Execution Time: 0.000005 seconds
Node: bert/embeddings/GatherV2, Execution Time: 0.000303 seconds
Node: bert/embeddings/Reshape_1, Execution Time: 0.000008 seconds
Node: bert/embeddings/one_hot, Execution Time: 0.000057 seconds
Node: bert/embeddings/MatMul, Execution Time: 0.058681 seconds
Node: bert/embeddings/Reshape_3, Execution Time: 0.000040 seconds
Node: bert/embeddings/add, Execution Time: 0.002191 seconds
Node: bert/embeddings/add_1, Execution Time: 0.000995 seconds
Node: bert/embeddings/LayerNorm/moments/mean, Execution Time: 0.005252 seconds
Node: bert/embeddings/LayerNorm/moments/SquaredDifference, Execution Time: 0.000705 seconds
Node: bert/embeddings/LayerNorm/moments/SquaredDifference__72, Execution Time: 0.000859 seconds
Node: bert/embeddings/LayerNorm/moments/variance, Execution Time: 0.000277 seconds
Node: bert/embeddings/LayerNorm/batchnorm/add, Execution Time: 0.000083 seconds
Node: bert/embeddings/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.009874 seconds
Node: bert/embeddings/LayerNorm/batchnorm/Rsqrt__74, Execution Time: 0.005224 seconds
Node: bert/embeddings/LayerNorm/batchnorm/mul, Execution Time: 0.000112 seconds
Node: bert/embeddings/LayerNorm/batchnorm/mul_2, Execution Time: 0.000082 seconds
Node: bert/embeddings/LayerNorm/batchnorm/sub, Execution Time: 0.000084 seconds
Node: bert/embeddings/LayerNorm/batchnorm/mul_1, Execution Time: 0.000651 seconds
Node: bert/embeddings/LayerNorm/batchnorm/add_1, Execution Time: 0.000699 seconds
Node: bert/encoder/Reshape_1, Execution Time: 0.000034 seconds
Matmul Fuse Node: bert/encoder/layer_0/attention/self/value/MatMul, Execution Time: 0.032293 seconds
Node: bert/encoder/layer_0/attention/self/Reshape_2, Execution Time: 0.000022 seconds
Node: bert/encoder/layer_0/attention/self/transpose_2, Execution Time: 0.000419 seconds
Matmul Fuse Node: bert/encoder/layer_0/attention/self/query/MatMul, Execution Time: 0.002810 seconds
Node: bert/encoder/layer_0/attention/self/Reshape, Execution Time: 0.000014 seconds
Node: bert/encoder/layer_0/attention/self/transpose, Execution Time: 0.000255 seconds
Matmul Fuse Node: bert/encoder/layer_0/attention/self/key/MatMul, Execution Time: 0.002757 seconds
Node: bert/encoder/layer_0/attention/self/Reshape_1, Execution Time: 0.000013 seconds
Node: bert/encoder/layer_0/attention/self/MatMul__306, Execution Time: 0.000244 seconds
Node: bert/encoder/layer_0/attention/self/MatMul, Execution Time: 0.004435 seconds
Node: bert/encoder/layer_0/attention/self/Mul, Execution Time: 0.001228 seconds
Node: bert/encoder/layer_0/attention/self/add, Execution Time: 0.001847 seconds
Node: bert/encoder/layer_0/attention/self/Softmax, Execution Time: 0.009442 seconds
Node: bert/encoder/layer_0/attention/self/MatMul_1, Execution Time: 0.001627 seconds
Node: bert/encoder/layer_0/attention/self/transpose_3, Execution Time: 0.000248 seconds
Node: bert/encoder/layer_0/attention/self/Reshape_3, Execution Time: 0.000039 seconds
Matmul Fuse Node: bert/encoder/layer_0/attention/output/dense/MatMul, Execution Time: 0.004302 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/moments/mean, Execution Time: 0.000221 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000308 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/moments/SquaredDifference__309, Execution Time: 0.000502 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/moments/variance, Execution Time: 0.000164 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000134 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000087 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/Rsqrt__311, Execution Time: 0.000123 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000124 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000094 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000082 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000573 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000687 seconds
Matmul Fuse Node: bert/encoder/layer_0/intermediate/dense/MatMul, Execution Time: 0.004891 seconds
Node: bert/encoder/layer_0/intermediate/dense/Pow, Execution Time: 0.017003 seconds
Node: bert/encoder/layer_0/intermediate/dense/mul, Execution Time: 0.001281 seconds
Node: bert/encoder/layer_0/intermediate/dense/add, Execution Time: 0.001519 seconds
Node: bert/encoder/layer_0/intermediate/dense/mul_1, Execution Time: 0.001138 seconds
Node: bert/encoder/layer_0/intermediate/dense/Tanh, Execution Time: 0.003386 seconds
Node: bert/encoder/layer_0/intermediate/dense/add_1, Execution Time: 0.001256 seconds
Node: bert/encoder/layer_0/intermediate/dense/mul_2, Execution Time: 0.001134 seconds
Node: bert/encoder/layer_0/intermediate/dense/mul_3, Execution Time: 0.001456 seconds
Matmul Fuse Node: bert/encoder/layer_0/output/dense/MatMul, Execution Time: 0.005001 seconds
Node: bert/encoder/layer_0/output/LayerNorm/moments/mean, Execution Time: 0.000202 seconds
Node: bert/encoder/layer_0/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000256 seconds
Node: bert/encoder/layer_0/output/LayerNorm/moments/SquaredDifference__313, Execution Time: 0.000330 seconds
Node: bert/encoder/layer_0/output/LayerNorm/moments/variance, Execution Time: 0.000158 seconds
Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/add, Execution Time: 0.000086 seconds
Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds
Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/Rsqrt__315, Execution Time: 0.000095 seconds
Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/mul, Execution Time: 0.000087 seconds
Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000078 seconds
Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/sub, Execution Time: 0.000082 seconds
Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000380 seconds
Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000453 seconds
Matmul Fuse Node: bert/encoder/layer_1/attention/self/value/MatMul, Execution Time: 0.002558 seconds
Node: bert/encoder/layer_1/attention/self/Reshape_2, Execution Time: 0.000022 seconds
Node: bert/encoder/layer_1/attention/self/transpose_2, Execution Time: 0.000253 seconds
Matmul Fuse Node: bert/encoder/layer_1/attention/self/query/MatMul, Execution Time: 0.002432 seconds
Node: bert/encoder/layer_1/attention/self/Reshape, Execution Time: 0.000013 seconds
Node: bert/encoder/layer_1/attention/self/transpose, Execution Time: 0.000263 seconds
Matmul Fuse Node: bert/encoder/layer_1/attention/self/key/MatMul, Execution Time: 0.002511 seconds
Node: bert/encoder/layer_1/attention/self/Reshape_1, Execution Time: 0.000012 seconds
Node: bert/encoder/layer_1/attention/self/MatMul__320, Execution Time: 0.000254 seconds
Node: bert/encoder/layer_1/attention/self/MatMul, Execution Time: 0.001446 seconds
Node: bert/encoder/layer_1/attention/self/Mul, Execution Time: 0.001190 seconds
Node: bert/encoder/layer_1/attention/self/add, Execution Time: 0.001764 seconds
Node: bert/encoder/layer_1/attention/self/Softmax, Execution Time: 0.001857 seconds
Node: bert/encoder/layer_1/attention/self/MatMul_1, Execution Time: 0.001093 seconds
Node: bert/encoder/layer_1/attention/self/transpose_3, Execution Time: 0.000275 seconds
Node: bert/encoder/layer_1/attention/self/Reshape_3, Execution Time: 0.000049 seconds
Matmul Fuse Node: bert/encoder/layer_1/attention/output/dense/MatMul, Execution Time: 0.003057 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/moments/mean, Execution Time: 0.000183 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000250 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/moments/SquaredDifference__323, Execution Time: 0.000631 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/moments/variance, Execution Time: 0.000168 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000091 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000062 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/Rsqrt__325, Execution Time: 0.000095 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000091 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000079 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000554 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000690 seconds
Matmul Fuse Node: bert/encoder/layer_1/intermediate/dense/MatMul, Execution Time: 0.004034 seconds
Node: bert/encoder/layer_1/intermediate/dense/Pow, Execution Time: 0.000731 seconds
Node: bert/encoder/layer_1/intermediate/dense/mul, Execution Time: 0.001221 seconds
Node: bert/encoder/layer_1/intermediate/dense/add, Execution Time: 0.001502 seconds
Node: bert/encoder/layer_1/intermediate/dense/mul_1, Execution Time: 0.001143 seconds
Node: bert/encoder/layer_1/intermediate/dense/Tanh, Execution Time: 0.001131 seconds
Node: bert/encoder/layer_1/intermediate/dense/add_1, Execution Time: 0.001182 seconds
Node: bert/encoder/layer_1/intermediate/dense/mul_2, Execution Time: 0.001253 seconds
Node: bert/encoder/layer_1/intermediate/dense/mul_3, Execution Time: 0.001813 seconds
Matmul Fuse Node: bert/encoder/layer_1/output/dense/MatMul, Execution Time: 0.006662 seconds
Node: bert/encoder/layer_1/output/LayerNorm/moments/mean, Execution Time: 0.000202 seconds
Node: bert/encoder/layer_1/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000252 seconds
Node: bert/encoder/layer_1/output/LayerNorm/moments/SquaredDifference__327, Execution Time: 0.000332 seconds
Node: bert/encoder/layer_1/output/LayerNorm/moments/variance, Execution Time: 0.000160 seconds
Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/add, Execution Time: 0.000102 seconds
Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000062 seconds
Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/Rsqrt__329, Execution Time: 0.000086 seconds
Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/mul, Execution Time: 0.000087 seconds
Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000078 seconds
Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/sub, Execution Time: 0.000081 seconds
Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000337 seconds
Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000443 seconds
Matmul Fuse Node: bert/encoder/layer_2/attention/self/value/MatMul, Execution Time: 0.002677 seconds
Node: bert/encoder/layer_2/attention/self/Reshape_2, Execution Time: 0.000021 seconds
Node: bert/encoder/layer_2/attention/self/transpose_2, Execution Time: 0.000255 seconds
Matmul Fuse Node: bert/encoder/layer_2/attention/self/query/MatMul, Execution Time: 0.002459 seconds
Node: bert/encoder/layer_2/attention/self/Reshape, Execution Time: 0.000013 seconds
Node: bert/encoder/layer_2/attention/self/transpose, Execution Time: 0.000248 seconds
Matmul Fuse Node: bert/encoder/layer_2/attention/self/key/MatMul, Execution Time: 0.002476 seconds
Node: bert/encoder/layer_2/attention/self/Reshape_1, Execution Time: 0.000013 seconds
Node: bert/encoder/layer_2/attention/self/MatMul__334, Execution Time: 0.000248 seconds
Node: bert/encoder/layer_2/attention/self/MatMul, Execution Time: 0.001387 seconds
Node: bert/encoder/layer_2/attention/self/Mul, Execution Time: 0.001173 seconds
Node: bert/encoder/layer_2/attention/self/add, Execution Time: 0.001809 seconds
Node: bert/encoder/layer_2/attention/self/Softmax, Execution Time: 0.001880 seconds
Node: bert/encoder/layer_2/attention/self/MatMul_1, Execution Time: 0.001072 seconds
Node: bert/encoder/layer_2/attention/self/transpose_3, Execution Time: 0.000247 seconds
Node: bert/encoder/layer_2/attention/self/Reshape_3, Execution Time: 0.000050 seconds
Matmul Fuse Node: bert/encoder/layer_2/attention/output/dense/MatMul, Execution Time: 0.003161 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/moments/mean, Execution Time: 0.000189 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000256 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/moments/SquaredDifference__337, Execution Time: 0.000484 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/moments/variance, Execution Time: 0.000168 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000087 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/Rsqrt__339, Execution Time: 0.000083 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000082 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000115 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000579 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000859 seconds
Matmul Fuse Node: bert/encoder/layer_2/intermediate/dense/MatMul, Execution Time: 0.004076 seconds
Node: bert/encoder/layer_2/intermediate/dense/Pow, Execution Time: 0.000725 seconds
Node: bert/encoder/layer_2/intermediate/dense/mul, Execution Time: 0.001148 seconds
Node: bert/encoder/layer_2/intermediate/dense/add, Execution Time: 0.001407 seconds
Node: bert/encoder/layer_2/intermediate/dense/mul_1, Execution Time: 0.001091 seconds
Node: bert/encoder/layer_2/intermediate/dense/Tanh, Execution Time: 0.001081 seconds
Node: bert/encoder/layer_2/intermediate/dense/add_1, Execution Time: 0.001105 seconds
Node: bert/encoder/layer_2/intermediate/dense/mul_2, Execution Time: 0.001135 seconds
Node: bert/encoder/layer_2/intermediate/dense/mul_3, Execution Time: 0.001367 seconds
Matmul Fuse Node: bert/encoder/layer_2/output/dense/MatMul, Execution Time: 0.004812 seconds
Node: bert/encoder/layer_2/output/LayerNorm/moments/mean, Execution Time: 0.000204 seconds
Node: bert/encoder/layer_2/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000255 seconds
Node: bert/encoder/layer_2/output/LayerNorm/moments/SquaredDifference__341, Execution Time: 0.000336 seconds
Node: bert/encoder/layer_2/output/LayerNorm/moments/variance, Execution Time: 0.000159 seconds
Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/add, Execution Time: 0.000088 seconds
Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds
Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/Rsqrt__343, Execution Time: 0.000078 seconds
Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/mul, Execution Time: 0.000085 seconds
Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000078 seconds
Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/sub, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000346 seconds
Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000448 seconds
Matmul Fuse Node: bert/encoder/layer_3/attention/self/value/MatMul, Execution Time: 0.002630 seconds
Node: bert/encoder/layer_3/attention/self/Reshape_2, Execution Time: 0.000021 seconds
Node: bert/encoder/layer_3/attention/self/transpose_2, Execution Time: 0.000256 seconds
Matmul Fuse Node: bert/encoder/layer_3/attention/self/query/MatMul, Execution Time: 0.002530 seconds
Node: bert/encoder/layer_3/attention/self/Reshape, Execution Time: 0.000016 seconds
Node: bert/encoder/layer_3/attention/self/transpose, Execution Time: 0.000256 seconds
Matmul Fuse Node: bert/encoder/layer_3/attention/self/key/MatMul, Execution Time: 0.003758 seconds
Node: bert/encoder/layer_3/attention/self/Reshape_1, Execution Time: 0.000018 seconds
Node: bert/encoder/layer_3/attention/self/MatMul__348, Execution Time: 0.000253 seconds
Node: bert/encoder/layer_3/attention/self/MatMul, Execution Time: 0.001421 seconds
Node: bert/encoder/layer_3/attention/self/Mul, Execution Time: 0.001252 seconds
Node: bert/encoder/layer_3/attention/self/add, Execution Time: 0.001809 seconds
Node: bert/encoder/layer_3/attention/self/Softmax, Execution Time: 0.001957 seconds
Node: bert/encoder/layer_3/attention/self/MatMul_1, Execution Time: 0.001081 seconds
Node: bert/encoder/layer_3/attention/self/transpose_3, Execution Time: 0.000248 seconds
Node: bert/encoder/layer_3/attention/self/Reshape_3, Execution Time: 0.000048 seconds
Matmul Fuse Node: bert/encoder/layer_3/attention/output/dense/MatMul, Execution Time: 0.003146 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/moments/mean, Execution Time: 0.000193 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000250 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/moments/SquaredDifference__351, Execution Time: 0.000336 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/moments/variance, Execution Time: 0.000157 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000082 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000064 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/Rsqrt__353, Execution Time: 0.000084 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000081 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000077 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000561 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000689 seconds
Matmul Fuse Node: bert/encoder/layer_3/intermediate/dense/MatMul, Execution Time: 0.003938 seconds
Node: bert/encoder/layer_3/intermediate/dense/Pow, Execution Time: 0.000794 seconds
Node: bert/encoder/layer_3/intermediate/dense/mul, Execution Time: 0.001151 seconds
Node: bert/encoder/layer_3/intermediate/dense/add, Execution Time: 0.001678 seconds
Node: bert/encoder/layer_3/intermediate/dense/mul_1, Execution Time: 0.001543 seconds
Node: bert/encoder/layer_3/intermediate/dense/Tanh, Execution Time: 0.001437 seconds
Node: bert/encoder/layer_3/intermediate/dense/add_1, Execution Time: 0.001474 seconds
Node: bert/encoder/layer_3/intermediate/dense/mul_2, Execution Time: 0.001465 seconds
Node: bert/encoder/layer_3/intermediate/dense/mul_3, Execution Time: 0.001801 seconds
Matmul Fuse Node: bert/encoder/layer_3/output/dense/MatMul, Execution Time: 0.004658 seconds
Node: bert/encoder/layer_3/output/LayerNorm/moments/mean, Execution Time: 0.000191 seconds
Node: bert/encoder/layer_3/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000251 seconds
Node: bert/encoder/layer_3/output/LayerNorm/moments/SquaredDifference__355, Execution Time: 0.000336 seconds
Node: bert/encoder/layer_3/output/LayerNorm/moments/variance, Execution Time: 0.000164 seconds
Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/add, Execution Time: 0.000102 seconds
Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000062 seconds
Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/Rsqrt__357, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/mul, Execution Time: 0.000118 seconds
Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000092 seconds
Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/sub, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000322 seconds
Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000440 seconds
Matmul Fuse Node: bert/encoder/layer_4/attention/self/value/MatMul, Execution Time: 0.002587 seconds
Node: bert/encoder/layer_4/attention/self/Reshape_2, Execution Time: 0.000021 seconds
Node: bert/encoder/layer_4/attention/self/transpose_2, Execution Time: 0.000255 seconds
Matmul Fuse Node: bert/encoder/layer_4/attention/self/query/MatMul, Execution Time: 0.002412 seconds
Node: bert/encoder/layer_4/attention/self/Reshape, Execution Time: 0.000013 seconds
Node: bert/encoder/layer_4/attention/self/transpose, Execution Time: 0.000253 seconds
Matmul Fuse Node: bert/encoder/layer_4/attention/self/key/MatMul, Execution Time: 0.002434 seconds
Node: bert/encoder/layer_4/attention/self/Reshape_1, Execution Time: 0.000012 seconds
Node: bert/encoder/layer_4/attention/self/MatMul__362, Execution Time: 0.000252 seconds
Node: bert/encoder/layer_4/attention/self/MatMul, Execution Time: 0.001425 seconds
Node: bert/encoder/layer_4/attention/self/Mul, Execution Time: 0.001194 seconds
Node: bert/encoder/layer_4/attention/self/add, Execution Time: 0.001842 seconds
Node: bert/encoder/layer_4/attention/self/Softmax, Execution Time: 0.001931 seconds
Node: bert/encoder/layer_4/attention/self/MatMul_1, Execution Time: 0.001112 seconds
Node: bert/encoder/layer_4/attention/self/transpose_3, Execution Time: 0.000232 seconds
Node: bert/encoder/layer_4/attention/self/Reshape_3, Execution Time: 0.000049 seconds
Matmul Fuse Node: bert/encoder/layer_4/attention/output/dense/MatMul, Execution Time: 0.003081 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/moments/mean, Execution Time: 0.000186 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000265 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/moments/SquaredDifference__365, Execution Time: 0.000471 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/moments/variance, Execution Time: 0.000160 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000085 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/Rsqrt__367, Execution Time: 0.000094 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000083 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000092 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000085 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000563 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000691 seconds
Matmul Fuse Node: bert/encoder/layer_4/intermediate/dense/MatMul, Execution Time: 0.004001 seconds
Node: bert/encoder/layer_4/intermediate/dense/Pow, Execution Time: 0.000753 seconds
Node: bert/encoder/layer_4/intermediate/dense/mul, Execution Time: 0.001149 seconds
Node: bert/encoder/layer_4/intermediate/dense/add, Execution Time: 0.001443 seconds
Node: bert/encoder/layer_4/intermediate/dense/mul_1, Execution Time: 0.001169 seconds
Node: bert/encoder/layer_4/intermediate/dense/Tanh, Execution Time: 0.001112 seconds
Node: bert/encoder/layer_4/intermediate/dense/add_1, Execution Time: 0.001204 seconds
Node: bert/encoder/layer_4/intermediate/dense/mul_2, Execution Time: 0.001200 seconds
Node: bert/encoder/layer_4/intermediate/dense/mul_3, Execution Time: 0.001414 seconds
Matmul Fuse Node: bert/encoder/layer_4/output/dense/MatMul, Execution Time: 0.004920 seconds
Node: bert/encoder/layer_4/output/LayerNorm/moments/mean, Execution Time: 0.000207 seconds
Node: bert/encoder/layer_4/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000252 seconds
Node: bert/encoder/layer_4/output/LayerNorm/moments/SquaredDifference__369, Execution Time: 0.000337 seconds
Node: bert/encoder/layer_4/output/LayerNorm/moments/variance, Execution Time: 0.000188 seconds
Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/add, Execution Time: 0.000092 seconds
Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds
Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/Rsqrt__371, Execution Time: 0.000079 seconds
Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/mul, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000076 seconds
Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/sub, Execution Time: 0.000084 seconds
Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000335 seconds
Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000436 seconds
Matmul Fuse Node: bert/encoder/layer_5/attention/self/value/MatMul, Execution Time: 0.002555 seconds
Node: bert/encoder/layer_5/attention/self/Reshape_2, Execution Time: 0.000022 seconds
Node: bert/encoder/layer_5/attention/self/transpose_2, Execution Time: 0.000259 seconds
Matmul Fuse Node: bert/encoder/layer_5/attention/self/query/MatMul, Execution Time: 0.002431 seconds
Node: bert/encoder/layer_5/attention/self/Reshape, Execution Time: 0.000014 seconds
Node: bert/encoder/layer_5/attention/self/transpose, Execution Time: 0.000251 seconds
Matmul Fuse Node: bert/encoder/layer_5/attention/self/key/MatMul, Execution Time: 0.002439 seconds
Node: bert/encoder/layer_5/attention/self/Reshape_1, Execution Time: 0.000014 seconds
Node: bert/encoder/layer_5/attention/self/MatMul__376, Execution Time: 0.000264 seconds
Node: bert/encoder/layer_5/attention/self/MatMul, Execution Time: 0.001407 seconds
Node: bert/encoder/layer_5/attention/self/Mul, Execution Time: 0.001152 seconds
Node: bert/encoder/layer_5/attention/self/add, Execution Time: 0.001769 seconds
Node: bert/encoder/layer_5/attention/self/Softmax, Execution Time: 0.001909 seconds
Node: bert/encoder/layer_5/attention/self/MatMul_1, Execution Time: 0.001083 seconds
Node: bert/encoder/layer_5/attention/self/transpose_3, Execution Time: 0.000246 seconds
Node: bert/encoder/layer_5/attention/self/Reshape_3, Execution Time: 0.000050 seconds
Matmul Fuse Node: bert/encoder/layer_5/attention/output/dense/MatMul, Execution Time: 0.003054 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/moments/mean, Execution Time: 0.000192 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000247 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/moments/SquaredDifference__379, Execution Time: 0.000465 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/moments/variance, Execution Time: 0.000162 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000084 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000058 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/Rsqrt__381, Execution Time: 0.000083 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000082 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000091 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000075 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000750 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000707 seconds
Matmul Fuse Node: bert/encoder/layer_5/intermediate/dense/MatMul, Execution Time: 0.004036 seconds
Node: bert/encoder/layer_5/intermediate/dense/Pow, Execution Time: 0.000737 seconds
Node: bert/encoder/layer_5/intermediate/dense/mul, Execution Time: 0.001198 seconds
Node: bert/encoder/layer_5/intermediate/dense/add, Execution Time: 0.001422 seconds
Node: bert/encoder/layer_5/intermediate/dense/mul_1, Execution Time: 0.001147 seconds
Node: bert/encoder/layer_5/intermediate/dense/Tanh, Execution Time: 0.001076 seconds
Node: bert/encoder/layer_5/intermediate/dense/add_1, Execution Time: 0.001121 seconds
Node: bert/encoder/layer_5/intermediate/dense/mul_2, Execution Time: 0.001108 seconds
Node: bert/encoder/layer_5/intermediate/dense/mul_3, Execution Time: 0.001373 seconds
Matmul Fuse Node: bert/encoder/layer_5/output/dense/MatMul, Execution Time: 0.004757 seconds
Node: bert/encoder/layer_5/output/LayerNorm/moments/mean, Execution Time: 0.000195 seconds
Node: bert/encoder/layer_5/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000257 seconds
Node: bert/encoder/layer_5/output/LayerNorm/moments/SquaredDifference__383, Execution Time: 0.000337 seconds
Node: bert/encoder/layer_5/output/LayerNorm/moments/variance, Execution Time: 0.000160 seconds
Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/add, Execution Time: 0.000086 seconds
Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds
Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/Rsqrt__385, Execution Time: 0.000083 seconds
Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/mul, Execution Time: 0.000079 seconds
Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000075 seconds
Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/sub, Execution Time: 0.000085 seconds
Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000338 seconds
Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000434 seconds
Matmul Fuse Node: bert/encoder/layer_6/attention/self/value/MatMul, Execution Time: 0.002607 seconds
Node: bert/encoder/layer_6/attention/self/Reshape_2, Execution Time: 0.000021 seconds
Node: bert/encoder/layer_6/attention/self/transpose_2, Execution Time: 0.000256 seconds
Matmul Fuse Node: bert/encoder/layer_6/attention/self/query/MatMul, Execution Time: 0.002414 seconds
Node: bert/encoder/layer_6/attention/self/Reshape, Execution Time: 0.000013 seconds
Node: bert/encoder/layer_6/attention/self/transpose, Execution Time: 0.000253 seconds
Matmul Fuse Node: bert/encoder/layer_6/attention/self/key/MatMul, Execution Time: 0.003550 seconds
Node: bert/encoder/layer_6/attention/self/Reshape_1, Execution Time: 0.000012 seconds
Node: bert/encoder/layer_6/attention/self/MatMul__390, Execution Time: 0.000250 seconds
Node: bert/encoder/layer_6/attention/self/MatMul, Execution Time: 0.001337 seconds
Node: bert/encoder/layer_6/attention/self/Mul, Execution Time: 0.001162 seconds
Node: bert/encoder/layer_6/attention/self/add, Execution Time: 0.001764 seconds
Node: bert/encoder/layer_6/attention/self/Softmax, Execution Time: 0.001960 seconds
Node: bert/encoder/layer_6/attention/self/MatMul_1, Execution Time: 0.001106 seconds
Node: bert/encoder/layer_6/attention/self/transpose_3, Execution Time: 0.000247 seconds
Node: bert/encoder/layer_6/attention/self/Reshape_3, Execution Time: 0.000051 seconds
Matmul Fuse Node: bert/encoder/layer_6/attention/output/dense/MatMul, Execution Time: 0.002967 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/moments/mean, Execution Time: 0.000190 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000257 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/moments/SquaredDifference__393, Execution Time: 0.000338 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/moments/variance, Execution Time: 0.000167 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000090 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/Rsqrt__395, Execution Time: 0.000081 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000082 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000087 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000085 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000341 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000719 seconds
Matmul Fuse Node: bert/encoder/layer_6/intermediate/dense/MatMul, Execution Time: 0.004126 seconds
Node: bert/encoder/layer_6/intermediate/dense/Pow, Execution Time: 0.000736 seconds
Node: bert/encoder/layer_6/intermediate/dense/mul, Execution Time: 0.001186 seconds
Node: bert/encoder/layer_6/intermediate/dense/add, Execution Time: 0.001434 seconds
Node: bert/encoder/layer_6/intermediate/dense/mul_1, Execution Time: 0.001187 seconds
Node: bert/encoder/layer_6/intermediate/dense/Tanh, Execution Time: 0.001077 seconds
Node: bert/encoder/layer_6/intermediate/dense/add_1, Execution Time: 0.001118 seconds
Node: bert/encoder/layer_6/intermediate/dense/mul_2, Execution Time: 0.001136 seconds
Node: bert/encoder/layer_6/intermediate/dense/mul_3, Execution Time: 0.001378 seconds
Matmul Fuse Node: bert/encoder/layer_6/output/dense/MatMul, Execution Time: 0.004672 seconds
Node: bert/encoder/layer_6/output/LayerNorm/moments/mean, Execution Time: 0.000201 seconds
Node: bert/encoder/layer_6/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000262 seconds
Node: bert/encoder/layer_6/output/LayerNorm/moments/SquaredDifference__397, Execution Time: 0.000332 seconds
Node: bert/encoder/layer_6/output/LayerNorm/moments/variance, Execution Time: 0.000160 seconds
Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/add, Execution Time: 0.000087 seconds
Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000072 seconds
Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/Rsqrt__399, Execution Time: 0.000096 seconds
Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/mul, Execution Time: 0.000092 seconds
Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000086 seconds
Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/sub, Execution Time: 0.000079 seconds
Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000335 seconds
Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000422 seconds
Matmul Fuse Node: bert/encoder/layer_7/attention/self/value/MatMul, Execution Time: 0.002581 seconds
Node: bert/encoder/layer_7/attention/self/Reshape_2, Execution Time: 0.000022 seconds
Node: bert/encoder/layer_7/attention/self/transpose_2, Execution Time: 0.000255 seconds
Matmul Fuse Node: bert/encoder/layer_7/attention/self/query/MatMul, Execution Time: 0.002487 seconds
Node: bert/encoder/layer_7/attention/self/Reshape, Execution Time: 0.000013 seconds
Node: bert/encoder/layer_7/attention/self/transpose, Execution Time: 0.000264 seconds
Matmul Fuse Node: bert/encoder/layer_7/attention/self/key/MatMul, Execution Time: 0.002403 seconds
Node: bert/encoder/layer_7/attention/self/Reshape_1, Execution Time: 0.000011 seconds
Node: bert/encoder/layer_7/attention/self/MatMul__404, Execution Time: 0.000245 seconds
Node: bert/encoder/layer_7/attention/self/MatMul, Execution Time: 0.001323 seconds
Node: bert/encoder/layer_7/attention/self/Mul, Execution Time: 0.001184 seconds
Node: bert/encoder/layer_7/attention/self/add, Execution Time: 0.001801 seconds
Node: bert/encoder/layer_7/attention/self/Softmax, Execution Time: 0.001961 seconds
Node: bert/encoder/layer_7/attention/self/MatMul_1, Execution Time: 0.001071 seconds
Node: bert/encoder/layer_7/attention/self/transpose_3, Execution Time: 0.000239 seconds
Node: bert/encoder/layer_7/attention/self/Reshape_3, Execution Time: 0.000051 seconds
Matmul Fuse Node: bert/encoder/layer_7/attention/output/dense/MatMul, Execution Time: 0.003001 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/moments/mean, Execution Time: 0.000187 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000248 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/moments/SquaredDifference__407, Execution Time: 0.000336 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/moments/variance, Execution Time: 0.000156 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000086 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000066 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/Rsqrt__409, Execution Time: 0.000079 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000077 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000079 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000627 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000680 seconds
Matmul Fuse Node: bert/encoder/layer_7/intermediate/dense/MatMul, Execution Time: 0.004050 seconds
Node: bert/encoder/layer_7/intermediate/dense/Pow, Execution Time: 0.000728 seconds
Node: bert/encoder/layer_7/intermediate/dense/mul, Execution Time: 0.001137 seconds
Node: bert/encoder/layer_7/intermediate/dense/add, Execution Time: 0.001427 seconds
Node: bert/encoder/layer_7/intermediate/dense/mul_1, Execution Time: 0.001146 seconds
Node: bert/encoder/layer_7/intermediate/dense/Tanh, Execution Time: 0.001064 seconds
Node: bert/encoder/layer_7/intermediate/dense/add_1, Execution Time: 0.001094 seconds
Node: bert/encoder/layer_7/intermediate/dense/mul_2, Execution Time: 0.001149 seconds
Node: bert/encoder/layer_7/intermediate/dense/mul_3, Execution Time: 0.001402 seconds
Matmul Fuse Node: bert/encoder/layer_7/output/dense/MatMul, Execution Time: 0.005224 seconds
Node: bert/encoder/layer_7/output/LayerNorm/moments/mean, Execution Time: 0.000263 seconds
Node: bert/encoder/layer_7/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000328 seconds
Node: bert/encoder/layer_7/output/LayerNorm/moments/SquaredDifference__411, Execution Time: 0.000437 seconds
Node: bert/encoder/layer_7/output/LayerNorm/moments/variance, Execution Time: 0.000203 seconds
Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/add, Execution Time: 0.000111 seconds
Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000076 seconds
Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/Rsqrt__413, Execution Time: 0.000099 seconds
Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/mul, Execution Time: 0.000095 seconds
Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000087 seconds
Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/sub, Execution Time: 0.000098 seconds
Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000414 seconds
Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000607 seconds
Matmul Fuse Node: bert/encoder/layer_8/attention/self/value/MatMul, Execution Time: 0.003144 seconds
Node: bert/encoder/layer_8/attention/self/Reshape_2, Execution Time: 0.000031 seconds
Node: bert/encoder/layer_8/attention/self/transpose_2, Execution Time: 0.000375 seconds
Matmul Fuse Node: bert/encoder/layer_8/attention/self/query/MatMul, Execution Time: 0.002753 seconds
Node: bert/encoder/layer_8/attention/self/Reshape, Execution Time: 0.000014 seconds
Node: bert/encoder/layer_8/attention/self/transpose, Execution Time: 0.000262 seconds
Matmul Fuse Node: bert/encoder/layer_8/attention/self/key/MatMul, Execution Time: 0.002415 seconds
Node: bert/encoder/layer_8/attention/self/Reshape_1, Execution Time: 0.000012 seconds
Node: bert/encoder/layer_8/attention/self/MatMul__418, Execution Time: 0.000247 seconds
Node: bert/encoder/layer_8/attention/self/MatMul, Execution Time: 0.001366 seconds
Node: bert/encoder/layer_8/attention/self/Mul, Execution Time: 0.001230 seconds
Node: bert/encoder/layer_8/attention/self/add, Execution Time: 0.001800 seconds
Node: bert/encoder/layer_8/attention/self/Softmax, Execution Time: 0.001901 seconds
Node: bert/encoder/layer_8/attention/self/MatMul_1, Execution Time: 0.001041 seconds
Node: bert/encoder/layer_8/attention/self/transpose_3, Execution Time: 0.000247 seconds
Node: bert/encoder/layer_8/attention/self/Reshape_3, Execution Time: 0.000046 seconds
Matmul Fuse Node: bert/encoder/layer_8/attention/output/dense/MatMul, Execution Time: 0.002922 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/moments/mean, Execution Time: 0.000186 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000256 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/moments/SquaredDifference__421, Execution Time: 0.000335 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/moments/variance, Execution Time: 0.000160 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000085 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000061 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/Rsqrt__423, Execution Time: 0.000078 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000081 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000097 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000083 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000572 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000684 seconds
Matmul Fuse Node: bert/encoder/layer_8/intermediate/dense/MatMul, Execution Time: 0.004099 seconds
Node: bert/encoder/layer_8/intermediate/dense/Pow, Execution Time: 0.000729 seconds
Node: bert/encoder/layer_8/intermediate/dense/mul, Execution Time: 0.001211 seconds
Node: bert/encoder/layer_8/intermediate/dense/add, Execution Time: 0.001483 seconds
Node: bert/encoder/layer_8/intermediate/dense/mul_1, Execution Time: 0.001129 seconds
Node: bert/encoder/layer_8/intermediate/dense/Tanh, Execution Time: 0.001110 seconds
Node: bert/encoder/layer_8/intermediate/dense/add_1, Execution Time: 0.001166 seconds
Node: bert/encoder/layer_8/intermediate/dense/mul_2, Execution Time: 0.001137 seconds
Node: bert/encoder/layer_8/intermediate/dense/mul_3, Execution Time: 0.001444 seconds
Matmul Fuse Node: bert/encoder/layer_8/output/dense/MatMul, Execution Time: 0.004842 seconds
Node: bert/encoder/layer_8/output/LayerNorm/moments/mean, Execution Time: 0.000192 seconds
Node: bert/encoder/layer_8/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000262 seconds
Node: bert/encoder/layer_8/output/LayerNorm/moments/SquaredDifference__425, Execution Time: 0.000331 seconds
Node: bert/encoder/layer_8/output/LayerNorm/moments/variance, Execution Time: 0.000162 seconds
Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/add, Execution Time: 0.000086 seconds
Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000059 seconds
Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/Rsqrt__427, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/mul, Execution Time: 0.000082 seconds
Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000102 seconds
Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/sub, Execution Time: 0.000083 seconds
Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000334 seconds
Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000436 seconds
Matmul Fuse Node: bert/encoder/layer_9/attention/self/value/MatMul, Execution Time: 0.002491 seconds
Node: bert/encoder/layer_9/attention/self/Reshape_2, Execution Time: 0.000021 seconds
Node: bert/encoder/layer_9/attention/self/transpose_2, Execution Time: 0.000252 seconds
Matmul Fuse Node: bert/encoder/layer_9/attention/self/query/MatMul, Execution Time: 0.002437 seconds
Node: bert/encoder/layer_9/attention/self/Reshape, Execution Time: 0.000013 seconds
Node: bert/encoder/layer_9/attention/self/transpose, Execution Time: 0.000260 seconds
Matmul Fuse Node: bert/encoder/layer_9/attention/self/key/MatMul, Execution Time: 0.003515 seconds
Node: bert/encoder/layer_9/attention/self/Reshape_1, Execution Time: 0.000012 seconds
Node: bert/encoder/layer_9/attention/self/MatMul__432, Execution Time: 0.000255 seconds
Node: bert/encoder/layer_9/attention/self/MatMul, Execution Time: 0.001313 seconds
Node: bert/encoder/layer_9/attention/self/Mul, Execution Time: 0.001149 seconds
Node: bert/encoder/layer_9/attention/self/add, Execution Time: 0.001719 seconds
Node: bert/encoder/layer_9/attention/self/Softmax, Execution Time: 0.001892 seconds
Node: bert/encoder/layer_9/attention/self/MatMul_1, Execution Time: 0.001084 seconds
Node: bert/encoder/layer_9/attention/self/transpose_3, Execution Time: 0.000239 seconds
Node: bert/encoder/layer_9/attention/self/Reshape_3, Execution Time: 0.000050 seconds
Matmul Fuse Node: bert/encoder/layer_9/attention/output/dense/MatMul, Execution Time: 0.002982 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/moments/mean, Execution Time: 0.000190 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000257 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/moments/SquaredDifference__435, Execution Time: 0.000335 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/moments/variance, Execution Time: 0.000159 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000087 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000064 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/Rsqrt__437, Execution Time: 0.000082 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000082 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000077 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000081 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000342 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000691 seconds
Matmul Fuse Node: bert/encoder/layer_9/intermediate/dense/MatMul, Execution Time: 0.004079 seconds
Node: bert/encoder/layer_9/intermediate/dense/Pow, Execution Time: 0.000738 seconds
Node: bert/encoder/layer_9/intermediate/dense/mul, Execution Time: 0.001164 seconds
Node: bert/encoder/layer_9/intermediate/dense/add, Execution Time: 0.001425 seconds
Node: bert/encoder/layer_9/intermediate/dense/mul_1, Execution Time: 0.001187 seconds
Node: bert/encoder/layer_9/intermediate/dense/Tanh, Execution Time: 0.001101 seconds
Node: bert/encoder/layer_9/intermediate/dense/add_1, Execution Time: 0.001185 seconds
Node: bert/encoder/layer_9/intermediate/dense/mul_2, Execution Time: 0.001150 seconds
Node: bert/encoder/layer_9/intermediate/dense/mul_3, Execution Time: 0.001426 seconds
Matmul Fuse Node: bert/encoder/layer_9/output/dense/MatMul, Execution Time: 0.006454 seconds
Node: bert/encoder/layer_9/output/LayerNorm/moments/mean, Execution Time: 0.000262 seconds
Node: bert/encoder/layer_9/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000336 seconds
Node: bert/encoder/layer_9/output/LayerNorm/moments/SquaredDifference__439, Execution Time: 0.000435 seconds
Node: bert/encoder/layer_9/output/LayerNorm/moments/variance, Execution Time: 0.000201 seconds
Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/add, Execution Time: 0.000108 seconds
Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/Rsqrt__441, Execution Time: 0.000107 seconds
Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/mul, Execution Time: 0.000097 seconds
Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000086 seconds
Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/sub, Execution Time: 0.000100 seconds
Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000428 seconds
Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000542 seconds
Matmul Fuse Node: bert/encoder/layer_10/attention/self/value/MatMul, Execution Time: 0.002448 seconds
Node: bert/encoder/layer_10/attention/self/Reshape_2, Execution Time: 0.000021 seconds
Node: bert/encoder/layer_10/attention/self/transpose_2, Execution Time: 0.000253 seconds
Matmul Fuse Node: bert/encoder/layer_10/attention/self/query/MatMul, Execution Time: 0.002485 seconds
Node: bert/encoder/layer_10/attention/self/Reshape, Execution Time: 0.000013 seconds
Node: bert/encoder/layer_10/attention/self/transpose, Execution Time: 0.000248 seconds
Matmul Fuse Node: bert/encoder/layer_10/attention/self/key/MatMul, Execution Time: 0.002360 seconds
Node: bert/encoder/layer_10/attention/self/Reshape_1, Execution Time: 0.000011 seconds
Node: bert/encoder/layer_10/attention/self/MatMul__446, Execution Time: 0.000256 seconds
Node: bert/encoder/layer_10/attention/self/MatMul, Execution Time: 0.001312 seconds
Node: bert/encoder/layer_10/attention/self/Mul, Execution Time: 0.001150 seconds
Node: bert/encoder/layer_10/attention/self/add, Execution Time: 0.001782 seconds
Node: bert/encoder/layer_10/attention/self/Softmax, Execution Time: 0.001897 seconds
Node: bert/encoder/layer_10/attention/self/MatMul_1, Execution Time: 0.001040 seconds
Node: bert/encoder/layer_10/attention/self/transpose_3, Execution Time: 0.000246 seconds
Node: bert/encoder/layer_10/attention/self/Reshape_3, Execution Time: 0.000055 seconds
Matmul Fuse Node: bert/encoder/layer_10/attention/output/dense/MatMul, Execution Time: 0.003088 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/moments/mean, Execution Time: 0.000187 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000251 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/moments/SquaredDifference__449, Execution Time: 0.000469 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/moments/variance, Execution Time: 0.000159 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000085 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/Rsqrt__451, Execution Time: 0.000085 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000079 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000076 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000086 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000572 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000695 seconds
Matmul Fuse Node: bert/encoder/layer_10/intermediate/dense/MatMul, Execution Time: 0.004106 seconds
Node: bert/encoder/layer_10/intermediate/dense/Pow, Execution Time: 0.000736 seconds
Node: bert/encoder/layer_10/intermediate/dense/mul, Execution Time: 0.001168 seconds
Node: bert/encoder/layer_10/intermediate/dense/add, Execution Time: 0.001451 seconds
Node: bert/encoder/layer_10/intermediate/dense/mul_1, Execution Time: 0.001139 seconds
Node: bert/encoder/layer_10/intermediate/dense/Tanh, Execution Time: 0.001075 seconds
Node: bert/encoder/layer_10/intermediate/dense/add_1, Execution Time: 0.001112 seconds
Node: bert/encoder/layer_10/intermediate/dense/mul_2, Execution Time: 0.001109 seconds
Node: bert/encoder/layer_10/intermediate/dense/mul_3, Execution Time: 0.001375 seconds
Matmul Fuse Node: bert/encoder/layer_10/output/dense/MatMul, Execution Time: 0.004691 seconds
Node: bert/encoder/layer_10/output/LayerNorm/moments/mean, Execution Time: 0.000202 seconds
Node: bert/encoder/layer_10/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000253 seconds
Node: bert/encoder/layer_10/output/LayerNorm/moments/SquaredDifference__453, Execution Time: 0.000340 seconds
Node: bert/encoder/layer_10/output/LayerNorm/moments/variance, Execution Time: 0.000159 seconds
Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/add, Execution Time: 0.000086 seconds
Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds
Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/Rsqrt__455, Execution Time: 0.000097 seconds
Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/mul, Execution Time: 0.000082 seconds
Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000093 seconds
Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/sub, Execution Time: 0.000087 seconds
Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000335 seconds
Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000422 seconds
Matmul Fuse Node: bert/encoder/layer_11/attention/self/value/MatMul, Execution Time: 0.002482 seconds
Node: bert/encoder/layer_11/attention/self/Reshape_2, Execution Time: 0.000021 seconds
Node: bert/encoder/layer_11/attention/self/transpose_2, Execution Time: 0.000258 seconds
Matmul Fuse Node: bert/encoder/layer_11/attention/self/query/MatMul, Execution Time: 0.002486 seconds
Node: bert/encoder/layer_11/attention/self/Reshape, Execution Time: 0.000013 seconds
Node: bert/encoder/layer_11/attention/self/transpose, Execution Time: 0.000248 seconds
Matmul Fuse Node: bert/encoder/layer_11/attention/self/key/MatMul, Execution Time: 0.002335 seconds
Node: bert/encoder/layer_11/attention/self/Reshape_1, Execution Time: 0.000011 seconds
Node: bert/encoder/layer_11/attention/self/MatMul__460, Execution Time: 0.000252 seconds
Node: bert/encoder/layer_11/attention/self/MatMul, Execution Time: 0.001331 seconds
Node: bert/encoder/layer_11/attention/self/Mul, Execution Time: 0.001167 seconds
Node: bert/encoder/layer_11/attention/self/add, Execution Time: 0.001809 seconds
Node: bert/encoder/layer_11/attention/self/Softmax, Execution Time: 0.002041 seconds
Node: bert/encoder/layer_11/attention/self/MatMul_1, Execution Time: 0.001040 seconds
Node: bert/encoder/layer_11/attention/self/transpose_3, Execution Time: 0.000237 seconds
Node: bert/encoder/layer_11/attention/self/Reshape_3, Execution Time: 0.000046 seconds
Matmul Fuse Node: bert/encoder/layer_11/attention/output/dense/MatMul, Execution Time: 0.002971 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/moments/mean, Execution Time: 0.000187 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000253 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/moments/SquaredDifference__463, Execution Time: 0.000474 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/moments/variance, Execution Time: 0.000160 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000084 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000059 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/Rsqrt__465, Execution Time: 0.000083 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000085 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000082 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000081 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000565 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000673 seconds
Matmul Fuse Node: bert/encoder/layer_11/intermediate/dense/MatMul, Execution Time: 0.004257 seconds
Node: bert/encoder/layer_11/intermediate/dense/Pow, Execution Time: 0.000738 seconds
Node: bert/encoder/layer_11/intermediate/dense/mul, Execution Time: 0.001146 seconds
Node: bert/encoder/layer_11/intermediate/dense/add, Execution Time: 0.001443 seconds
Node: bert/encoder/layer_11/intermediate/dense/mul_1, Execution Time: 0.001164 seconds
Node: bert/encoder/layer_11/intermediate/dense/Tanh, Execution Time: 0.001117 seconds
Node: bert/encoder/layer_11/intermediate/dense/add_1, Execution Time: 0.001119 seconds
Node: bert/encoder/layer_11/intermediate/dense/mul_2, Execution Time: 0.001105 seconds
Node: bert/encoder/layer_11/intermediate/dense/mul_3, Execution Time: 0.001384 seconds
Matmul Fuse Node: bert/encoder/layer_11/output/dense/MatMul, Execution Time: 0.005082 seconds
Node: bert/encoder/layer_11/output/LayerNorm/moments/mean, Execution Time: 0.000202 seconds
Node: bert/encoder/layer_11/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000252 seconds
Node: bert/encoder/layer_11/output/LayerNorm/moments/SquaredDifference__467, Execution Time: 0.000330 seconds
Node: bert/encoder/layer_11/output/LayerNorm/moments/variance, Execution Time: 0.000159 seconds
Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/add, Execution Time: 0.000088 seconds
Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000061 seconds
Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/Rsqrt__469, Execution Time: 0.000083 seconds
Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/mul, Execution Time: 0.000102 seconds
Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/sub, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000325 seconds
Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000442 seconds
Matmul Fuse Node: MatMul, Execution Time: 0.002985 seconds
Node: Reshape_1, Execution Time: 0.000021 seconds
Node: transpose, Execution Time: 0.000081 seconds
Node: unstack, Execution Time: 0.000046 seconds
Node: unstack__490, Execution Time: 0.000005 seconds
Node: unstack__488, Execution Time: 0.000006 seconds

Total Execution Time: 0.704871 seconds

Total Matmul Fuse Execution Time: 0.276326 seconds

Total Matmul + Add Execution Time: 0.441405 seconds
Execution complete.
Editor is loading...
Leave a Comment