Untitled
unknown
c_cpp
a month ago
79 kB
1
Indexable
Model Input Name: unique_ids_raw_output___9:0, Shape: [0] Model Input Name: segment_ids:0, Shape: [0, 256] Model Input Name: input_mask:0, Shape: [0, 256] Model Input Name: input_ids:0, Shape: [0, 256] Starting model execution... Inputs Details: Input Name: input_ids:0 Shape: (1, 256) Data (first 10 values): [14558 12889 27202 3336 19210 22185 18020 28836 27440 30487]... -------------------------------------------------- Input Name: segment_ids:0 Shape: (1, 256) Data (first 10 values): [0 1 1 0 1 0 0 0 1 1]... -------------------------------------------------- Input Name: input_mask:0 Shape: (1, 256) Data (first 10 values): [0 0 1 0 0 0 0 1 1 0]... -------------------------------------------------- Input Name: unique_ids_raw_output___9:0 Shape: (0,) Data (first 10 values): []... -------------------------------------------------- No Add node related to MatMul output: bert/embeddings/MatMul. Executing regular MatMul. Fusing MatMul with Add for Node: bert/encoder/layer_0/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_0/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_0/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_0/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_0/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_0/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_0/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_0/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_0/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_0/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_0/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_0/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_0/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_0/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_0/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_0/output/add Fusing MatMul with Add for Node: bert/encoder/layer_1/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_1/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_1/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_1/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_1/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_1/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_1/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_1/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_1/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_1/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_1/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_1/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_1/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_1/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_1/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_1/output/add Fusing MatMul with Add for Node: bert/encoder/layer_2/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_2/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_2/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_2/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_2/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_2/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_2/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_2/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_2/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_2/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_2/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_2/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_2/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_2/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_2/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_2/output/add Fusing MatMul with Add for Node: bert/encoder/layer_3/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_3/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_3/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_3/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_3/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_3/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_3/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_3/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_3/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_3/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_3/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_3/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_3/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_3/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_3/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_3/output/add Fusing MatMul with Add for Node: bert/encoder/layer_4/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_4/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_4/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_4/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_4/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_4/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_4/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_4/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_4/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_4/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_4/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_4/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_4/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_4/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_4/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_4/output/add Fusing MatMul with Add for Node: bert/encoder/layer_5/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_5/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_5/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_5/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_5/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_5/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_5/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_5/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_5/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_5/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_5/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_5/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_5/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_5/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_5/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_5/output/add Fusing MatMul with Add for Node: bert/encoder/layer_6/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_6/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_6/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_6/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_6/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_6/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_6/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_6/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_6/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_6/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_6/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_6/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_6/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_6/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_6/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_6/output/add Fusing MatMul with Add for Node: bert/encoder/layer_7/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_7/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_7/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_7/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_7/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_7/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_7/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_7/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_7/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_7/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_7/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_7/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_7/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_7/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_7/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_7/output/add Fusing MatMul with Add for Node: bert/encoder/layer_8/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_8/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_8/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_8/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_8/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_8/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_8/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_8/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_8/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_8/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_8/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_8/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_8/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_8/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_8/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_8/output/add Fusing MatMul with Add for Node: bert/encoder/layer_9/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_9/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_9/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_9/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_9/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_9/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_9/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_9/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_9/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_9/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_9/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_9/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_9/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_9/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_9/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_9/output/add Fusing MatMul with Add for Node: bert/encoder/layer_10/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_10/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_10/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_10/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_10/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_10/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_10/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_10/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_10/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_10/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_10/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_10/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_10/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_10/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_10/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_10/output/add Fusing MatMul with Add for Node: bert/encoder/layer_11/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_11/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_11/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_11/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_11/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_11/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_11/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_11/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_11/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_11/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_11/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_11/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_11/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_11/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_11/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_11/output/add Fusing MatMul with Add for Node: MatMul Skipping already processed Add Node: BiasAdd Node Execution Times: Node: unique_ids_graph_outputs_Identity__10, Execution Time: 0.000005 seconds Node: bert/encoder/Shape, Execution Time: 0.000003 seconds Node: bert/encoder/Shape__12, Execution Time: 0.000009 seconds Node: bert/encoder/strided_slice, Execution Time: 0.000072 seconds Node: bert/encoder/strided_slice__16, Execution Time: 0.000005 seconds Node: bert/encoder/strided_slice__17, Execution Time: 0.000005 seconds Node: bert/encoder/ones/packed_Unsqueeze__18, Execution Time: 0.000010 seconds Node: bert/encoder/ones/packed_Concat__21, Execution Time: 0.000010 seconds Node: bert/encoder/ones__22, Execution Time: 0.000004 seconds Node: bert/encoder/ones, Execution Time: 0.000010 seconds Node: bert/encoder/Reshape, Execution Time: 0.000006 seconds Node: bert/encoder/Cast, Execution Time: 0.000004 seconds Node: bert/encoder/mul, Execution Time: 0.028060 seconds Node: bert/encoder/layer_9/attention/self/ExpandDims, Execution Time: 0.000035 seconds Node: bert/encoder/layer_9/attention/self/sub, Execution Time: 0.006613 seconds Node: bert/encoder/layer_9/attention/self/mul_1, Execution Time: 0.000350 seconds Node: bert/embeddings/Reshape_2, Execution Time: 0.000030 seconds Node: bert/embeddings/Reshape, Execution Time: 0.000005 seconds Node: bert/embeddings/GatherV2, Execution Time: 0.000303 seconds Node: bert/embeddings/Reshape_1, Execution Time: 0.000008 seconds Node: bert/embeddings/one_hot, Execution Time: 0.000057 seconds Node: bert/embeddings/MatMul, Execution Time: 0.058681 seconds Node: bert/embeddings/Reshape_3, Execution Time: 0.000040 seconds Node: bert/embeddings/add, Execution Time: 0.002191 seconds Node: bert/embeddings/add_1, Execution Time: 0.000995 seconds Node: bert/embeddings/LayerNorm/moments/mean, Execution Time: 0.005252 seconds Node: bert/embeddings/LayerNorm/moments/SquaredDifference, Execution Time: 0.000705 seconds Node: bert/embeddings/LayerNorm/moments/SquaredDifference__72, Execution Time: 0.000859 seconds Node: bert/embeddings/LayerNorm/moments/variance, Execution Time: 0.000277 seconds Node: bert/embeddings/LayerNorm/batchnorm/add, Execution Time: 0.000083 seconds Node: bert/embeddings/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.009874 seconds Node: bert/embeddings/LayerNorm/batchnorm/Rsqrt__74, Execution Time: 0.005224 seconds Node: bert/embeddings/LayerNorm/batchnorm/mul, Execution Time: 0.000112 seconds Node: bert/embeddings/LayerNorm/batchnorm/mul_2, Execution Time: 0.000082 seconds Node: bert/embeddings/LayerNorm/batchnorm/sub, Execution Time: 0.000084 seconds Node: bert/embeddings/LayerNorm/batchnorm/mul_1, Execution Time: 0.000651 seconds Node: bert/embeddings/LayerNorm/batchnorm/add_1, Execution Time: 0.000699 seconds Node: bert/encoder/Reshape_1, Execution Time: 0.000034 seconds Matmul Fuse Node: bert/encoder/layer_0/attention/self/value/MatMul, Execution Time: 0.032293 seconds Node: bert/encoder/layer_0/attention/self/Reshape_2, Execution Time: 0.000022 seconds Node: bert/encoder/layer_0/attention/self/transpose_2, Execution Time: 0.000419 seconds Matmul Fuse Node: bert/encoder/layer_0/attention/self/query/MatMul, Execution Time: 0.002810 seconds Node: bert/encoder/layer_0/attention/self/Reshape, Execution Time: 0.000014 seconds Node: bert/encoder/layer_0/attention/self/transpose, Execution Time: 0.000255 seconds Matmul Fuse Node: bert/encoder/layer_0/attention/self/key/MatMul, Execution Time: 0.002757 seconds Node: bert/encoder/layer_0/attention/self/Reshape_1, Execution Time: 0.000013 seconds Node: bert/encoder/layer_0/attention/self/MatMul__306, Execution Time: 0.000244 seconds Node: bert/encoder/layer_0/attention/self/MatMul, Execution Time: 0.004435 seconds Node: bert/encoder/layer_0/attention/self/Mul, Execution Time: 0.001228 seconds Node: bert/encoder/layer_0/attention/self/add, Execution Time: 0.001847 seconds Node: bert/encoder/layer_0/attention/self/Softmax, Execution Time: 0.009442 seconds Node: bert/encoder/layer_0/attention/self/MatMul_1, Execution Time: 0.001627 seconds Node: bert/encoder/layer_0/attention/self/transpose_3, Execution Time: 0.000248 seconds Node: bert/encoder/layer_0/attention/self/Reshape_3, Execution Time: 0.000039 seconds Matmul Fuse Node: bert/encoder/layer_0/attention/output/dense/MatMul, Execution Time: 0.004302 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/moments/mean, Execution Time: 0.000221 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000308 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/moments/SquaredDifference__309, Execution Time: 0.000502 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/moments/variance, Execution Time: 0.000164 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000134 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000087 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/Rsqrt__311, Execution Time: 0.000123 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000124 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000094 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000082 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000573 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000687 seconds Matmul Fuse Node: bert/encoder/layer_0/intermediate/dense/MatMul, Execution Time: 0.004891 seconds Node: bert/encoder/layer_0/intermediate/dense/Pow, Execution Time: 0.017003 seconds Node: bert/encoder/layer_0/intermediate/dense/mul, Execution Time: 0.001281 seconds Node: bert/encoder/layer_0/intermediate/dense/add, Execution Time: 0.001519 seconds Node: bert/encoder/layer_0/intermediate/dense/mul_1, Execution Time: 0.001138 seconds Node: bert/encoder/layer_0/intermediate/dense/Tanh, Execution Time: 0.003386 seconds Node: bert/encoder/layer_0/intermediate/dense/add_1, Execution Time: 0.001256 seconds Node: bert/encoder/layer_0/intermediate/dense/mul_2, Execution Time: 0.001134 seconds Node: bert/encoder/layer_0/intermediate/dense/mul_3, Execution Time: 0.001456 seconds Matmul Fuse Node: bert/encoder/layer_0/output/dense/MatMul, Execution Time: 0.005001 seconds Node: bert/encoder/layer_0/output/LayerNorm/moments/mean, Execution Time: 0.000202 seconds Node: bert/encoder/layer_0/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000256 seconds Node: bert/encoder/layer_0/output/LayerNorm/moments/SquaredDifference__313, Execution Time: 0.000330 seconds Node: bert/encoder/layer_0/output/LayerNorm/moments/variance, Execution Time: 0.000158 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/add, Execution Time: 0.000086 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/Rsqrt__315, Execution Time: 0.000095 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/mul, Execution Time: 0.000087 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000078 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/sub, Execution Time: 0.000082 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000380 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000453 seconds Matmul Fuse Node: bert/encoder/layer_1/attention/self/value/MatMul, Execution Time: 0.002558 seconds Node: bert/encoder/layer_1/attention/self/Reshape_2, Execution Time: 0.000022 seconds Node: bert/encoder/layer_1/attention/self/transpose_2, Execution Time: 0.000253 seconds Matmul Fuse Node: bert/encoder/layer_1/attention/self/query/MatMul, Execution Time: 0.002432 seconds Node: bert/encoder/layer_1/attention/self/Reshape, Execution Time: 0.000013 seconds Node: bert/encoder/layer_1/attention/self/transpose, Execution Time: 0.000263 seconds Matmul Fuse Node: bert/encoder/layer_1/attention/self/key/MatMul, Execution Time: 0.002511 seconds Node: bert/encoder/layer_1/attention/self/Reshape_1, Execution Time: 0.000012 seconds Node: bert/encoder/layer_1/attention/self/MatMul__320, Execution Time: 0.000254 seconds Node: bert/encoder/layer_1/attention/self/MatMul, Execution Time: 0.001446 seconds Node: bert/encoder/layer_1/attention/self/Mul, Execution Time: 0.001190 seconds Node: bert/encoder/layer_1/attention/self/add, Execution Time: 0.001764 seconds Node: bert/encoder/layer_1/attention/self/Softmax, Execution Time: 0.001857 seconds Node: bert/encoder/layer_1/attention/self/MatMul_1, Execution Time: 0.001093 seconds Node: bert/encoder/layer_1/attention/self/transpose_3, Execution Time: 0.000275 seconds Node: bert/encoder/layer_1/attention/self/Reshape_3, Execution Time: 0.000049 seconds Matmul Fuse Node: bert/encoder/layer_1/attention/output/dense/MatMul, Execution Time: 0.003057 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/moments/mean, Execution Time: 0.000183 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000250 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/moments/SquaredDifference__323, Execution Time: 0.000631 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/moments/variance, Execution Time: 0.000168 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000091 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000062 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/Rsqrt__325, Execution Time: 0.000095 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000080 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000091 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000079 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000554 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000690 seconds Matmul Fuse Node: bert/encoder/layer_1/intermediate/dense/MatMul, Execution Time: 0.004034 seconds Node: bert/encoder/layer_1/intermediate/dense/Pow, Execution Time: 0.000731 seconds Node: bert/encoder/layer_1/intermediate/dense/mul, Execution Time: 0.001221 seconds Node: bert/encoder/layer_1/intermediate/dense/add, Execution Time: 0.001502 seconds Node: bert/encoder/layer_1/intermediate/dense/mul_1, Execution Time: 0.001143 seconds Node: bert/encoder/layer_1/intermediate/dense/Tanh, Execution Time: 0.001131 seconds Node: bert/encoder/layer_1/intermediate/dense/add_1, Execution Time: 0.001182 seconds Node: bert/encoder/layer_1/intermediate/dense/mul_2, Execution Time: 0.001253 seconds Node: bert/encoder/layer_1/intermediate/dense/mul_3, Execution Time: 0.001813 seconds Matmul Fuse Node: bert/encoder/layer_1/output/dense/MatMul, Execution Time: 0.006662 seconds Node: bert/encoder/layer_1/output/LayerNorm/moments/mean, Execution Time: 0.000202 seconds Node: bert/encoder/layer_1/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000252 seconds Node: bert/encoder/layer_1/output/LayerNorm/moments/SquaredDifference__327, Execution Time: 0.000332 seconds Node: bert/encoder/layer_1/output/LayerNorm/moments/variance, Execution Time: 0.000160 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/add, Execution Time: 0.000102 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000062 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/Rsqrt__329, Execution Time: 0.000086 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/mul, Execution Time: 0.000087 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000078 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/sub, Execution Time: 0.000081 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000337 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000443 seconds Matmul Fuse Node: bert/encoder/layer_2/attention/self/value/MatMul, Execution Time: 0.002677 seconds Node: bert/encoder/layer_2/attention/self/Reshape_2, Execution Time: 0.000021 seconds Node: bert/encoder/layer_2/attention/self/transpose_2, Execution Time: 0.000255 seconds Matmul Fuse Node: bert/encoder/layer_2/attention/self/query/MatMul, Execution Time: 0.002459 seconds Node: bert/encoder/layer_2/attention/self/Reshape, Execution Time: 0.000013 seconds Node: bert/encoder/layer_2/attention/self/transpose, Execution Time: 0.000248 seconds Matmul Fuse Node: bert/encoder/layer_2/attention/self/key/MatMul, Execution Time: 0.002476 seconds Node: bert/encoder/layer_2/attention/self/Reshape_1, Execution Time: 0.000013 seconds Node: bert/encoder/layer_2/attention/self/MatMul__334, Execution Time: 0.000248 seconds Node: bert/encoder/layer_2/attention/self/MatMul, Execution Time: 0.001387 seconds Node: bert/encoder/layer_2/attention/self/Mul, Execution Time: 0.001173 seconds Node: bert/encoder/layer_2/attention/self/add, Execution Time: 0.001809 seconds Node: bert/encoder/layer_2/attention/self/Softmax, Execution Time: 0.001880 seconds Node: bert/encoder/layer_2/attention/self/MatMul_1, Execution Time: 0.001072 seconds Node: bert/encoder/layer_2/attention/self/transpose_3, Execution Time: 0.000247 seconds Node: bert/encoder/layer_2/attention/self/Reshape_3, Execution Time: 0.000050 seconds Matmul Fuse Node: bert/encoder/layer_2/attention/output/dense/MatMul, Execution Time: 0.003161 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/moments/mean, Execution Time: 0.000189 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000256 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/moments/SquaredDifference__337, Execution Time: 0.000484 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/moments/variance, Execution Time: 0.000168 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000087 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/Rsqrt__339, Execution Time: 0.000083 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000082 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000115 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000080 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000579 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000859 seconds Matmul Fuse Node: bert/encoder/layer_2/intermediate/dense/MatMul, Execution Time: 0.004076 seconds Node: bert/encoder/layer_2/intermediate/dense/Pow, Execution Time: 0.000725 seconds Node: bert/encoder/layer_2/intermediate/dense/mul, Execution Time: 0.001148 seconds Node: bert/encoder/layer_2/intermediate/dense/add, Execution Time: 0.001407 seconds Node: bert/encoder/layer_2/intermediate/dense/mul_1, Execution Time: 0.001091 seconds Node: bert/encoder/layer_2/intermediate/dense/Tanh, Execution Time: 0.001081 seconds Node: bert/encoder/layer_2/intermediate/dense/add_1, Execution Time: 0.001105 seconds Node: bert/encoder/layer_2/intermediate/dense/mul_2, Execution Time: 0.001135 seconds Node: bert/encoder/layer_2/intermediate/dense/mul_3, Execution Time: 0.001367 seconds Matmul Fuse Node: bert/encoder/layer_2/output/dense/MatMul, Execution Time: 0.004812 seconds Node: bert/encoder/layer_2/output/LayerNorm/moments/mean, Execution Time: 0.000204 seconds Node: bert/encoder/layer_2/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000255 seconds Node: bert/encoder/layer_2/output/LayerNorm/moments/SquaredDifference__341, Execution Time: 0.000336 seconds Node: bert/encoder/layer_2/output/LayerNorm/moments/variance, Execution Time: 0.000159 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/add, Execution Time: 0.000088 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/Rsqrt__343, Execution Time: 0.000078 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/mul, Execution Time: 0.000085 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000078 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/sub, Execution Time: 0.000080 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000346 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000448 seconds Matmul Fuse Node: bert/encoder/layer_3/attention/self/value/MatMul, Execution Time: 0.002630 seconds Node: bert/encoder/layer_3/attention/self/Reshape_2, Execution Time: 0.000021 seconds Node: bert/encoder/layer_3/attention/self/transpose_2, Execution Time: 0.000256 seconds Matmul Fuse Node: bert/encoder/layer_3/attention/self/query/MatMul, Execution Time: 0.002530 seconds Node: bert/encoder/layer_3/attention/self/Reshape, Execution Time: 0.000016 seconds Node: bert/encoder/layer_3/attention/self/transpose, Execution Time: 0.000256 seconds Matmul Fuse Node: bert/encoder/layer_3/attention/self/key/MatMul, Execution Time: 0.003758 seconds Node: bert/encoder/layer_3/attention/self/Reshape_1, Execution Time: 0.000018 seconds Node: bert/encoder/layer_3/attention/self/MatMul__348, Execution Time: 0.000253 seconds Node: bert/encoder/layer_3/attention/self/MatMul, Execution Time: 0.001421 seconds Node: bert/encoder/layer_3/attention/self/Mul, Execution Time: 0.001252 seconds Node: bert/encoder/layer_3/attention/self/add, Execution Time: 0.001809 seconds Node: bert/encoder/layer_3/attention/self/Softmax, Execution Time: 0.001957 seconds Node: bert/encoder/layer_3/attention/self/MatMul_1, Execution Time: 0.001081 seconds Node: bert/encoder/layer_3/attention/self/transpose_3, Execution Time: 0.000248 seconds Node: bert/encoder/layer_3/attention/self/Reshape_3, Execution Time: 0.000048 seconds Matmul Fuse Node: bert/encoder/layer_3/attention/output/dense/MatMul, Execution Time: 0.003146 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/moments/mean, Execution Time: 0.000193 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000250 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/moments/SquaredDifference__351, Execution Time: 0.000336 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/moments/variance, Execution Time: 0.000157 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000082 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000064 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/Rsqrt__353, Execution Time: 0.000084 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000081 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000077 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000080 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000561 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000689 seconds Matmul Fuse Node: bert/encoder/layer_3/intermediate/dense/MatMul, Execution Time: 0.003938 seconds Node: bert/encoder/layer_3/intermediate/dense/Pow, Execution Time: 0.000794 seconds Node: bert/encoder/layer_3/intermediate/dense/mul, Execution Time: 0.001151 seconds Node: bert/encoder/layer_3/intermediate/dense/add, Execution Time: 0.001678 seconds Node: bert/encoder/layer_3/intermediate/dense/mul_1, Execution Time: 0.001543 seconds Node: bert/encoder/layer_3/intermediate/dense/Tanh, Execution Time: 0.001437 seconds Node: bert/encoder/layer_3/intermediate/dense/add_1, Execution Time: 0.001474 seconds Node: bert/encoder/layer_3/intermediate/dense/mul_2, Execution Time: 0.001465 seconds Node: bert/encoder/layer_3/intermediate/dense/mul_3, Execution Time: 0.001801 seconds Matmul Fuse Node: bert/encoder/layer_3/output/dense/MatMul, Execution Time: 0.004658 seconds Node: bert/encoder/layer_3/output/LayerNorm/moments/mean, Execution Time: 0.000191 seconds Node: bert/encoder/layer_3/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000251 seconds Node: bert/encoder/layer_3/output/LayerNorm/moments/SquaredDifference__355, Execution Time: 0.000336 seconds Node: bert/encoder/layer_3/output/LayerNorm/moments/variance, Execution Time: 0.000164 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/add, Execution Time: 0.000102 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000062 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/Rsqrt__357, Execution Time: 0.000080 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/mul, Execution Time: 0.000118 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000092 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/sub, Execution Time: 0.000080 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000322 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000440 seconds Matmul Fuse Node: bert/encoder/layer_4/attention/self/value/MatMul, Execution Time: 0.002587 seconds Node: bert/encoder/layer_4/attention/self/Reshape_2, Execution Time: 0.000021 seconds Node: bert/encoder/layer_4/attention/self/transpose_2, Execution Time: 0.000255 seconds Matmul Fuse Node: bert/encoder/layer_4/attention/self/query/MatMul, Execution Time: 0.002412 seconds Node: bert/encoder/layer_4/attention/self/Reshape, Execution Time: 0.000013 seconds Node: bert/encoder/layer_4/attention/self/transpose, Execution Time: 0.000253 seconds Matmul Fuse Node: bert/encoder/layer_4/attention/self/key/MatMul, Execution Time: 0.002434 seconds Node: bert/encoder/layer_4/attention/self/Reshape_1, Execution Time: 0.000012 seconds Node: bert/encoder/layer_4/attention/self/MatMul__362, Execution Time: 0.000252 seconds Node: bert/encoder/layer_4/attention/self/MatMul, Execution Time: 0.001425 seconds Node: bert/encoder/layer_4/attention/self/Mul, Execution Time: 0.001194 seconds Node: bert/encoder/layer_4/attention/self/add, Execution Time: 0.001842 seconds Node: bert/encoder/layer_4/attention/self/Softmax, Execution Time: 0.001931 seconds Node: bert/encoder/layer_4/attention/self/MatMul_1, Execution Time: 0.001112 seconds Node: bert/encoder/layer_4/attention/self/transpose_3, Execution Time: 0.000232 seconds Node: bert/encoder/layer_4/attention/self/Reshape_3, Execution Time: 0.000049 seconds Matmul Fuse Node: bert/encoder/layer_4/attention/output/dense/MatMul, Execution Time: 0.003081 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/moments/mean, Execution Time: 0.000186 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000265 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/moments/SquaredDifference__365, Execution Time: 0.000471 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/moments/variance, Execution Time: 0.000160 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000085 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/Rsqrt__367, Execution Time: 0.000094 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000083 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000092 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000085 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000563 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000691 seconds Matmul Fuse Node: bert/encoder/layer_4/intermediate/dense/MatMul, Execution Time: 0.004001 seconds Node: bert/encoder/layer_4/intermediate/dense/Pow, Execution Time: 0.000753 seconds Node: bert/encoder/layer_4/intermediate/dense/mul, Execution Time: 0.001149 seconds Node: bert/encoder/layer_4/intermediate/dense/add, Execution Time: 0.001443 seconds Node: bert/encoder/layer_4/intermediate/dense/mul_1, Execution Time: 0.001169 seconds Node: bert/encoder/layer_4/intermediate/dense/Tanh, Execution Time: 0.001112 seconds Node: bert/encoder/layer_4/intermediate/dense/add_1, Execution Time: 0.001204 seconds Node: bert/encoder/layer_4/intermediate/dense/mul_2, Execution Time: 0.001200 seconds Node: bert/encoder/layer_4/intermediate/dense/mul_3, Execution Time: 0.001414 seconds Matmul Fuse Node: bert/encoder/layer_4/output/dense/MatMul, Execution Time: 0.004920 seconds Node: bert/encoder/layer_4/output/LayerNorm/moments/mean, Execution Time: 0.000207 seconds Node: bert/encoder/layer_4/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000252 seconds Node: bert/encoder/layer_4/output/LayerNorm/moments/SquaredDifference__369, Execution Time: 0.000337 seconds Node: bert/encoder/layer_4/output/LayerNorm/moments/variance, Execution Time: 0.000188 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/add, Execution Time: 0.000092 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/Rsqrt__371, Execution Time: 0.000079 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/mul, Execution Time: 0.000080 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000076 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/sub, Execution Time: 0.000084 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000335 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000436 seconds Matmul Fuse Node: bert/encoder/layer_5/attention/self/value/MatMul, Execution Time: 0.002555 seconds Node: bert/encoder/layer_5/attention/self/Reshape_2, Execution Time: 0.000022 seconds Node: bert/encoder/layer_5/attention/self/transpose_2, Execution Time: 0.000259 seconds Matmul Fuse Node: bert/encoder/layer_5/attention/self/query/MatMul, Execution Time: 0.002431 seconds Node: bert/encoder/layer_5/attention/self/Reshape, Execution Time: 0.000014 seconds Node: bert/encoder/layer_5/attention/self/transpose, Execution Time: 0.000251 seconds Matmul Fuse Node: bert/encoder/layer_5/attention/self/key/MatMul, Execution Time: 0.002439 seconds Node: bert/encoder/layer_5/attention/self/Reshape_1, Execution Time: 0.000014 seconds Node: bert/encoder/layer_5/attention/self/MatMul__376, Execution Time: 0.000264 seconds Node: bert/encoder/layer_5/attention/self/MatMul, Execution Time: 0.001407 seconds Node: bert/encoder/layer_5/attention/self/Mul, Execution Time: 0.001152 seconds Node: bert/encoder/layer_5/attention/self/add, Execution Time: 0.001769 seconds Node: bert/encoder/layer_5/attention/self/Softmax, Execution Time: 0.001909 seconds Node: bert/encoder/layer_5/attention/self/MatMul_1, Execution Time: 0.001083 seconds Node: bert/encoder/layer_5/attention/self/transpose_3, Execution Time: 0.000246 seconds Node: bert/encoder/layer_5/attention/self/Reshape_3, Execution Time: 0.000050 seconds Matmul Fuse Node: bert/encoder/layer_5/attention/output/dense/MatMul, Execution Time: 0.003054 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/moments/mean, Execution Time: 0.000192 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000247 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/moments/SquaredDifference__379, Execution Time: 0.000465 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/moments/variance, Execution Time: 0.000162 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000084 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000058 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/Rsqrt__381, Execution Time: 0.000083 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000082 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000091 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000075 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000750 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000707 seconds Matmul Fuse Node: bert/encoder/layer_5/intermediate/dense/MatMul, Execution Time: 0.004036 seconds Node: bert/encoder/layer_5/intermediate/dense/Pow, Execution Time: 0.000737 seconds Node: bert/encoder/layer_5/intermediate/dense/mul, Execution Time: 0.001198 seconds Node: bert/encoder/layer_5/intermediate/dense/add, Execution Time: 0.001422 seconds Node: bert/encoder/layer_5/intermediate/dense/mul_1, Execution Time: 0.001147 seconds Node: bert/encoder/layer_5/intermediate/dense/Tanh, Execution Time: 0.001076 seconds Node: bert/encoder/layer_5/intermediate/dense/add_1, Execution Time: 0.001121 seconds Node: bert/encoder/layer_5/intermediate/dense/mul_2, Execution Time: 0.001108 seconds Node: bert/encoder/layer_5/intermediate/dense/mul_3, Execution Time: 0.001373 seconds Matmul Fuse Node: bert/encoder/layer_5/output/dense/MatMul, Execution Time: 0.004757 seconds Node: bert/encoder/layer_5/output/LayerNorm/moments/mean, Execution Time: 0.000195 seconds Node: bert/encoder/layer_5/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000257 seconds Node: bert/encoder/layer_5/output/LayerNorm/moments/SquaredDifference__383, Execution Time: 0.000337 seconds Node: bert/encoder/layer_5/output/LayerNorm/moments/variance, Execution Time: 0.000160 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/add, Execution Time: 0.000086 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/Rsqrt__385, Execution Time: 0.000083 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/mul, Execution Time: 0.000079 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000075 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/sub, Execution Time: 0.000085 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000338 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000434 seconds Matmul Fuse Node: bert/encoder/layer_6/attention/self/value/MatMul, Execution Time: 0.002607 seconds Node: bert/encoder/layer_6/attention/self/Reshape_2, Execution Time: 0.000021 seconds Node: bert/encoder/layer_6/attention/self/transpose_2, Execution Time: 0.000256 seconds Matmul Fuse Node: bert/encoder/layer_6/attention/self/query/MatMul, Execution Time: 0.002414 seconds Node: bert/encoder/layer_6/attention/self/Reshape, Execution Time: 0.000013 seconds Node: bert/encoder/layer_6/attention/self/transpose, Execution Time: 0.000253 seconds Matmul Fuse Node: bert/encoder/layer_6/attention/self/key/MatMul, Execution Time: 0.003550 seconds Node: bert/encoder/layer_6/attention/self/Reshape_1, Execution Time: 0.000012 seconds Node: bert/encoder/layer_6/attention/self/MatMul__390, Execution Time: 0.000250 seconds Node: bert/encoder/layer_6/attention/self/MatMul, Execution Time: 0.001337 seconds Node: bert/encoder/layer_6/attention/self/Mul, Execution Time: 0.001162 seconds Node: bert/encoder/layer_6/attention/self/add, Execution Time: 0.001764 seconds Node: bert/encoder/layer_6/attention/self/Softmax, Execution Time: 0.001960 seconds Node: bert/encoder/layer_6/attention/self/MatMul_1, Execution Time: 0.001106 seconds Node: bert/encoder/layer_6/attention/self/transpose_3, Execution Time: 0.000247 seconds Node: bert/encoder/layer_6/attention/self/Reshape_3, Execution Time: 0.000051 seconds Matmul Fuse Node: bert/encoder/layer_6/attention/output/dense/MatMul, Execution Time: 0.002967 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/moments/mean, Execution Time: 0.000190 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000257 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/moments/SquaredDifference__393, Execution Time: 0.000338 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/moments/variance, Execution Time: 0.000167 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000090 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/Rsqrt__395, Execution Time: 0.000081 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000082 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000087 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000085 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000341 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000719 seconds Matmul Fuse Node: bert/encoder/layer_6/intermediate/dense/MatMul, Execution Time: 0.004126 seconds Node: bert/encoder/layer_6/intermediate/dense/Pow, Execution Time: 0.000736 seconds Node: bert/encoder/layer_6/intermediate/dense/mul, Execution Time: 0.001186 seconds Node: bert/encoder/layer_6/intermediate/dense/add, Execution Time: 0.001434 seconds Node: bert/encoder/layer_6/intermediate/dense/mul_1, Execution Time: 0.001187 seconds Node: bert/encoder/layer_6/intermediate/dense/Tanh, Execution Time: 0.001077 seconds Node: bert/encoder/layer_6/intermediate/dense/add_1, Execution Time: 0.001118 seconds Node: bert/encoder/layer_6/intermediate/dense/mul_2, Execution Time: 0.001136 seconds Node: bert/encoder/layer_6/intermediate/dense/mul_3, Execution Time: 0.001378 seconds Matmul Fuse Node: bert/encoder/layer_6/output/dense/MatMul, Execution Time: 0.004672 seconds Node: bert/encoder/layer_6/output/LayerNorm/moments/mean, Execution Time: 0.000201 seconds Node: bert/encoder/layer_6/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000262 seconds Node: bert/encoder/layer_6/output/LayerNorm/moments/SquaredDifference__397, Execution Time: 0.000332 seconds Node: bert/encoder/layer_6/output/LayerNorm/moments/variance, Execution Time: 0.000160 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/add, Execution Time: 0.000087 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000072 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/Rsqrt__399, Execution Time: 0.000096 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/mul, Execution Time: 0.000092 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000086 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/sub, Execution Time: 0.000079 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000335 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000422 seconds Matmul Fuse Node: bert/encoder/layer_7/attention/self/value/MatMul, Execution Time: 0.002581 seconds Node: bert/encoder/layer_7/attention/self/Reshape_2, Execution Time: 0.000022 seconds Node: bert/encoder/layer_7/attention/self/transpose_2, Execution Time: 0.000255 seconds Matmul Fuse Node: bert/encoder/layer_7/attention/self/query/MatMul, Execution Time: 0.002487 seconds Node: bert/encoder/layer_7/attention/self/Reshape, Execution Time: 0.000013 seconds Node: bert/encoder/layer_7/attention/self/transpose, Execution Time: 0.000264 seconds Matmul Fuse Node: bert/encoder/layer_7/attention/self/key/MatMul, Execution Time: 0.002403 seconds Node: bert/encoder/layer_7/attention/self/Reshape_1, Execution Time: 0.000011 seconds Node: bert/encoder/layer_7/attention/self/MatMul__404, Execution Time: 0.000245 seconds Node: bert/encoder/layer_7/attention/self/MatMul, Execution Time: 0.001323 seconds Node: bert/encoder/layer_7/attention/self/Mul, Execution Time: 0.001184 seconds Node: bert/encoder/layer_7/attention/self/add, Execution Time: 0.001801 seconds Node: bert/encoder/layer_7/attention/self/Softmax, Execution Time: 0.001961 seconds Node: bert/encoder/layer_7/attention/self/MatMul_1, Execution Time: 0.001071 seconds Node: bert/encoder/layer_7/attention/self/transpose_3, Execution Time: 0.000239 seconds Node: bert/encoder/layer_7/attention/self/Reshape_3, Execution Time: 0.000051 seconds Matmul Fuse Node: bert/encoder/layer_7/attention/output/dense/MatMul, Execution Time: 0.003001 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/moments/mean, Execution Time: 0.000187 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000248 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/moments/SquaredDifference__407, Execution Time: 0.000336 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/moments/variance, Execution Time: 0.000156 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000086 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000066 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/Rsqrt__409, Execution Time: 0.000079 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000080 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000077 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000079 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000627 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000680 seconds Matmul Fuse Node: bert/encoder/layer_7/intermediate/dense/MatMul, Execution Time: 0.004050 seconds Node: bert/encoder/layer_7/intermediate/dense/Pow, Execution Time: 0.000728 seconds Node: bert/encoder/layer_7/intermediate/dense/mul, Execution Time: 0.001137 seconds Node: bert/encoder/layer_7/intermediate/dense/add, Execution Time: 0.001427 seconds Node: bert/encoder/layer_7/intermediate/dense/mul_1, Execution Time: 0.001146 seconds Node: bert/encoder/layer_7/intermediate/dense/Tanh, Execution Time: 0.001064 seconds Node: bert/encoder/layer_7/intermediate/dense/add_1, Execution Time: 0.001094 seconds Node: bert/encoder/layer_7/intermediate/dense/mul_2, Execution Time: 0.001149 seconds Node: bert/encoder/layer_7/intermediate/dense/mul_3, Execution Time: 0.001402 seconds Matmul Fuse Node: bert/encoder/layer_7/output/dense/MatMul, Execution Time: 0.005224 seconds Node: bert/encoder/layer_7/output/LayerNorm/moments/mean, Execution Time: 0.000263 seconds Node: bert/encoder/layer_7/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000328 seconds Node: bert/encoder/layer_7/output/LayerNorm/moments/SquaredDifference__411, Execution Time: 0.000437 seconds Node: bert/encoder/layer_7/output/LayerNorm/moments/variance, Execution Time: 0.000203 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/add, Execution Time: 0.000111 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000076 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/Rsqrt__413, Execution Time: 0.000099 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/mul, Execution Time: 0.000095 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000087 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/sub, Execution Time: 0.000098 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000414 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000607 seconds Matmul Fuse Node: bert/encoder/layer_8/attention/self/value/MatMul, Execution Time: 0.003144 seconds Node: bert/encoder/layer_8/attention/self/Reshape_2, Execution Time: 0.000031 seconds Node: bert/encoder/layer_8/attention/self/transpose_2, Execution Time: 0.000375 seconds Matmul Fuse Node: bert/encoder/layer_8/attention/self/query/MatMul, Execution Time: 0.002753 seconds Node: bert/encoder/layer_8/attention/self/Reshape, Execution Time: 0.000014 seconds Node: bert/encoder/layer_8/attention/self/transpose, Execution Time: 0.000262 seconds Matmul Fuse Node: bert/encoder/layer_8/attention/self/key/MatMul, Execution Time: 0.002415 seconds Node: bert/encoder/layer_8/attention/self/Reshape_1, Execution Time: 0.000012 seconds Node: bert/encoder/layer_8/attention/self/MatMul__418, Execution Time: 0.000247 seconds Node: bert/encoder/layer_8/attention/self/MatMul, Execution Time: 0.001366 seconds Node: bert/encoder/layer_8/attention/self/Mul, Execution Time: 0.001230 seconds Node: bert/encoder/layer_8/attention/self/add, Execution Time: 0.001800 seconds Node: bert/encoder/layer_8/attention/self/Softmax, Execution Time: 0.001901 seconds Node: bert/encoder/layer_8/attention/self/MatMul_1, Execution Time: 0.001041 seconds Node: bert/encoder/layer_8/attention/self/transpose_3, Execution Time: 0.000247 seconds Node: bert/encoder/layer_8/attention/self/Reshape_3, Execution Time: 0.000046 seconds Matmul Fuse Node: bert/encoder/layer_8/attention/output/dense/MatMul, Execution Time: 0.002922 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/moments/mean, Execution Time: 0.000186 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000256 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/moments/SquaredDifference__421, Execution Time: 0.000335 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/moments/variance, Execution Time: 0.000160 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000085 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000061 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/Rsqrt__423, Execution Time: 0.000078 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000081 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000097 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000083 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000572 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000684 seconds Matmul Fuse Node: bert/encoder/layer_8/intermediate/dense/MatMul, Execution Time: 0.004099 seconds Node: bert/encoder/layer_8/intermediate/dense/Pow, Execution Time: 0.000729 seconds Node: bert/encoder/layer_8/intermediate/dense/mul, Execution Time: 0.001211 seconds Node: bert/encoder/layer_8/intermediate/dense/add, Execution Time: 0.001483 seconds Node: bert/encoder/layer_8/intermediate/dense/mul_1, Execution Time: 0.001129 seconds Node: bert/encoder/layer_8/intermediate/dense/Tanh, Execution Time: 0.001110 seconds Node: bert/encoder/layer_8/intermediate/dense/add_1, Execution Time: 0.001166 seconds Node: bert/encoder/layer_8/intermediate/dense/mul_2, Execution Time: 0.001137 seconds Node: bert/encoder/layer_8/intermediate/dense/mul_3, Execution Time: 0.001444 seconds Matmul Fuse Node: bert/encoder/layer_8/output/dense/MatMul, Execution Time: 0.004842 seconds Node: bert/encoder/layer_8/output/LayerNorm/moments/mean, Execution Time: 0.000192 seconds Node: bert/encoder/layer_8/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000262 seconds Node: bert/encoder/layer_8/output/LayerNorm/moments/SquaredDifference__425, Execution Time: 0.000331 seconds Node: bert/encoder/layer_8/output/LayerNorm/moments/variance, Execution Time: 0.000162 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/add, Execution Time: 0.000086 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000059 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/Rsqrt__427, Execution Time: 0.000080 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/mul, Execution Time: 0.000082 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000102 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/sub, Execution Time: 0.000083 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000334 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000436 seconds Matmul Fuse Node: bert/encoder/layer_9/attention/self/value/MatMul, Execution Time: 0.002491 seconds Node: bert/encoder/layer_9/attention/self/Reshape_2, Execution Time: 0.000021 seconds Node: bert/encoder/layer_9/attention/self/transpose_2, Execution Time: 0.000252 seconds Matmul Fuse Node: bert/encoder/layer_9/attention/self/query/MatMul, Execution Time: 0.002437 seconds Node: bert/encoder/layer_9/attention/self/Reshape, Execution Time: 0.000013 seconds Node: bert/encoder/layer_9/attention/self/transpose, Execution Time: 0.000260 seconds Matmul Fuse Node: bert/encoder/layer_9/attention/self/key/MatMul, Execution Time: 0.003515 seconds Node: bert/encoder/layer_9/attention/self/Reshape_1, Execution Time: 0.000012 seconds Node: bert/encoder/layer_9/attention/self/MatMul__432, Execution Time: 0.000255 seconds Node: bert/encoder/layer_9/attention/self/MatMul, Execution Time: 0.001313 seconds Node: bert/encoder/layer_9/attention/self/Mul, Execution Time: 0.001149 seconds Node: bert/encoder/layer_9/attention/self/add, Execution Time: 0.001719 seconds Node: bert/encoder/layer_9/attention/self/Softmax, Execution Time: 0.001892 seconds Node: bert/encoder/layer_9/attention/self/MatMul_1, Execution Time: 0.001084 seconds Node: bert/encoder/layer_9/attention/self/transpose_3, Execution Time: 0.000239 seconds Node: bert/encoder/layer_9/attention/self/Reshape_3, Execution Time: 0.000050 seconds Matmul Fuse Node: bert/encoder/layer_9/attention/output/dense/MatMul, Execution Time: 0.002982 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/moments/mean, Execution Time: 0.000190 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000257 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/moments/SquaredDifference__435, Execution Time: 0.000335 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/moments/variance, Execution Time: 0.000159 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000087 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000064 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/Rsqrt__437, Execution Time: 0.000082 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000082 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000077 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000081 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000342 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000691 seconds Matmul Fuse Node: bert/encoder/layer_9/intermediate/dense/MatMul, Execution Time: 0.004079 seconds Node: bert/encoder/layer_9/intermediate/dense/Pow, Execution Time: 0.000738 seconds Node: bert/encoder/layer_9/intermediate/dense/mul, Execution Time: 0.001164 seconds Node: bert/encoder/layer_9/intermediate/dense/add, Execution Time: 0.001425 seconds Node: bert/encoder/layer_9/intermediate/dense/mul_1, Execution Time: 0.001187 seconds Node: bert/encoder/layer_9/intermediate/dense/Tanh, Execution Time: 0.001101 seconds Node: bert/encoder/layer_9/intermediate/dense/add_1, Execution Time: 0.001185 seconds Node: bert/encoder/layer_9/intermediate/dense/mul_2, Execution Time: 0.001150 seconds Node: bert/encoder/layer_9/intermediate/dense/mul_3, Execution Time: 0.001426 seconds Matmul Fuse Node: bert/encoder/layer_9/output/dense/MatMul, Execution Time: 0.006454 seconds Node: bert/encoder/layer_9/output/LayerNorm/moments/mean, Execution Time: 0.000262 seconds Node: bert/encoder/layer_9/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000336 seconds Node: bert/encoder/layer_9/output/LayerNorm/moments/SquaredDifference__439, Execution Time: 0.000435 seconds Node: bert/encoder/layer_9/output/LayerNorm/moments/variance, Execution Time: 0.000201 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/add, Execution Time: 0.000108 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000080 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/Rsqrt__441, Execution Time: 0.000107 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/mul, Execution Time: 0.000097 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000086 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/sub, Execution Time: 0.000100 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000428 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000542 seconds Matmul Fuse Node: bert/encoder/layer_10/attention/self/value/MatMul, Execution Time: 0.002448 seconds Node: bert/encoder/layer_10/attention/self/Reshape_2, Execution Time: 0.000021 seconds Node: bert/encoder/layer_10/attention/self/transpose_2, Execution Time: 0.000253 seconds Matmul Fuse Node: bert/encoder/layer_10/attention/self/query/MatMul, Execution Time: 0.002485 seconds Node: bert/encoder/layer_10/attention/self/Reshape, Execution Time: 0.000013 seconds Node: bert/encoder/layer_10/attention/self/transpose, Execution Time: 0.000248 seconds Matmul Fuse Node: bert/encoder/layer_10/attention/self/key/MatMul, Execution Time: 0.002360 seconds Node: bert/encoder/layer_10/attention/self/Reshape_1, Execution Time: 0.000011 seconds Node: bert/encoder/layer_10/attention/self/MatMul__446, Execution Time: 0.000256 seconds Node: bert/encoder/layer_10/attention/self/MatMul, Execution Time: 0.001312 seconds Node: bert/encoder/layer_10/attention/self/Mul, Execution Time: 0.001150 seconds Node: bert/encoder/layer_10/attention/self/add, Execution Time: 0.001782 seconds Node: bert/encoder/layer_10/attention/self/Softmax, Execution Time: 0.001897 seconds Node: bert/encoder/layer_10/attention/self/MatMul_1, Execution Time: 0.001040 seconds Node: bert/encoder/layer_10/attention/self/transpose_3, Execution Time: 0.000246 seconds Node: bert/encoder/layer_10/attention/self/Reshape_3, Execution Time: 0.000055 seconds Matmul Fuse Node: bert/encoder/layer_10/attention/output/dense/MatMul, Execution Time: 0.003088 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/moments/mean, Execution Time: 0.000187 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000251 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/moments/SquaredDifference__449, Execution Time: 0.000469 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/moments/variance, Execution Time: 0.000159 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000085 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/Rsqrt__451, Execution Time: 0.000085 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000079 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000076 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000086 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000572 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000695 seconds Matmul Fuse Node: bert/encoder/layer_10/intermediate/dense/MatMul, Execution Time: 0.004106 seconds Node: bert/encoder/layer_10/intermediate/dense/Pow, Execution Time: 0.000736 seconds Node: bert/encoder/layer_10/intermediate/dense/mul, Execution Time: 0.001168 seconds Node: bert/encoder/layer_10/intermediate/dense/add, Execution Time: 0.001451 seconds Node: bert/encoder/layer_10/intermediate/dense/mul_1, Execution Time: 0.001139 seconds Node: bert/encoder/layer_10/intermediate/dense/Tanh, Execution Time: 0.001075 seconds Node: bert/encoder/layer_10/intermediate/dense/add_1, Execution Time: 0.001112 seconds Node: bert/encoder/layer_10/intermediate/dense/mul_2, Execution Time: 0.001109 seconds Node: bert/encoder/layer_10/intermediate/dense/mul_3, Execution Time: 0.001375 seconds Matmul Fuse Node: bert/encoder/layer_10/output/dense/MatMul, Execution Time: 0.004691 seconds Node: bert/encoder/layer_10/output/LayerNorm/moments/mean, Execution Time: 0.000202 seconds Node: bert/encoder/layer_10/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000253 seconds Node: bert/encoder/layer_10/output/LayerNorm/moments/SquaredDifference__453, Execution Time: 0.000340 seconds Node: bert/encoder/layer_10/output/LayerNorm/moments/variance, Execution Time: 0.000159 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/add, Execution Time: 0.000086 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/Rsqrt__455, Execution Time: 0.000097 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/mul, Execution Time: 0.000082 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000093 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/sub, Execution Time: 0.000087 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000335 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000422 seconds Matmul Fuse Node: bert/encoder/layer_11/attention/self/value/MatMul, Execution Time: 0.002482 seconds Node: bert/encoder/layer_11/attention/self/Reshape_2, Execution Time: 0.000021 seconds Node: bert/encoder/layer_11/attention/self/transpose_2, Execution Time: 0.000258 seconds Matmul Fuse Node: bert/encoder/layer_11/attention/self/query/MatMul, Execution Time: 0.002486 seconds Node: bert/encoder/layer_11/attention/self/Reshape, Execution Time: 0.000013 seconds Node: bert/encoder/layer_11/attention/self/transpose, Execution Time: 0.000248 seconds Matmul Fuse Node: bert/encoder/layer_11/attention/self/key/MatMul, Execution Time: 0.002335 seconds Node: bert/encoder/layer_11/attention/self/Reshape_1, Execution Time: 0.000011 seconds Node: bert/encoder/layer_11/attention/self/MatMul__460, Execution Time: 0.000252 seconds Node: bert/encoder/layer_11/attention/self/MatMul, Execution Time: 0.001331 seconds Node: bert/encoder/layer_11/attention/self/Mul, Execution Time: 0.001167 seconds Node: bert/encoder/layer_11/attention/self/add, Execution Time: 0.001809 seconds Node: bert/encoder/layer_11/attention/self/Softmax, Execution Time: 0.002041 seconds Node: bert/encoder/layer_11/attention/self/MatMul_1, Execution Time: 0.001040 seconds Node: bert/encoder/layer_11/attention/self/transpose_3, Execution Time: 0.000237 seconds Node: bert/encoder/layer_11/attention/self/Reshape_3, Execution Time: 0.000046 seconds Matmul Fuse Node: bert/encoder/layer_11/attention/output/dense/MatMul, Execution Time: 0.002971 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/moments/mean, Execution Time: 0.000187 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000253 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/moments/SquaredDifference__463, Execution Time: 0.000474 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/moments/variance, Execution Time: 0.000160 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000084 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000059 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/Rsqrt__465, Execution Time: 0.000083 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000085 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000082 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000081 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000565 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000673 seconds Matmul Fuse Node: bert/encoder/layer_11/intermediate/dense/MatMul, Execution Time: 0.004257 seconds Node: bert/encoder/layer_11/intermediate/dense/Pow, Execution Time: 0.000738 seconds Node: bert/encoder/layer_11/intermediate/dense/mul, Execution Time: 0.001146 seconds Node: bert/encoder/layer_11/intermediate/dense/add, Execution Time: 0.001443 seconds Node: bert/encoder/layer_11/intermediate/dense/mul_1, Execution Time: 0.001164 seconds Node: bert/encoder/layer_11/intermediate/dense/Tanh, Execution Time: 0.001117 seconds Node: bert/encoder/layer_11/intermediate/dense/add_1, Execution Time: 0.001119 seconds Node: bert/encoder/layer_11/intermediate/dense/mul_2, Execution Time: 0.001105 seconds Node: bert/encoder/layer_11/intermediate/dense/mul_3, Execution Time: 0.001384 seconds Matmul Fuse Node: bert/encoder/layer_11/output/dense/MatMul, Execution Time: 0.005082 seconds Node: bert/encoder/layer_11/output/LayerNorm/moments/mean, Execution Time: 0.000202 seconds Node: bert/encoder/layer_11/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000252 seconds Node: bert/encoder/layer_11/output/LayerNorm/moments/SquaredDifference__467, Execution Time: 0.000330 seconds Node: bert/encoder/layer_11/output/LayerNorm/moments/variance, Execution Time: 0.000159 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/add, Execution Time: 0.000088 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000061 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/Rsqrt__469, Execution Time: 0.000083 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/mul, Execution Time: 0.000102 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000080 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/sub, Execution Time: 0.000080 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000325 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000442 seconds Matmul Fuse Node: MatMul, Execution Time: 0.002985 seconds Node: Reshape_1, Execution Time: 0.000021 seconds Node: transpose, Execution Time: 0.000081 seconds Node: unstack, Execution Time: 0.000046 seconds Node: unstack__490, Execution Time: 0.000005 seconds Node: unstack__488, Execution Time: 0.000006 seconds Total Execution Time: 0.704871 seconds Total Matmul Fuse Execution Time: 0.276326 seconds Total Matmul + Add Execution Time: 0.441405 seconds Execution complete.
Editor is loading...
Leave a Comment