Untitled
unknown
c_cpp
a month ago
146 kB
1
Indexable
Model Input Name: unique_ids_raw_output___9:0, Shape: [0] Model Input Name: segment_ids:0, Shape: [0, 256] Model Input Name: input_mask:0, Shape: [0, 256] Model Input Name: input_ids:0, Shape: [0, 256] Starting model execution... Inputs Details: Input Name: input_ids:0 Shape: (1, 256) Data (first 10 values): [28148 26736 988 2528 1043 3349 8281 12138 2763 27770]... -------------------------------------------------- Input Name: segment_ids:0 Shape: (1, 256) Data (first 10 values): [0 1 0 0 0 0 1 0 1 0]... -------------------------------------------------- Input Name: input_mask:0 Shape: (1, 256) Data (first 10 values): [1 1 0 0 1 0 1 0 1 1]... -------------------------------------------------- Input Name: unique_ids_raw_output___9:0 Shape: (0,) Data (first 10 values): []... -------------------------------------------------- No Add node related to MatMul output: bert/embeddings/MatMul. Executing regular MatMul. Fusing MatMul with Add for Node: bert/encoder/layer_0/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_0/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_0/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_0/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_0/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_0/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_0/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_0/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_0/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_0/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_0/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_0/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_0/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_0/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_0/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_0/output/add Fusing MatMul with Add for Node: bert/encoder/layer_1/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_1/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_1/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_1/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_1/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_1/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_1/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_1/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_1/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_1/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_1/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_1/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_1/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_1/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_1/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_1/output/add Fusing MatMul with Add for Node: bert/encoder/layer_2/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_2/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_2/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_2/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_2/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_2/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_2/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_2/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_2/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_2/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_2/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_2/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_2/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_2/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_2/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_2/output/add Fusing MatMul with Add for Node: bert/encoder/layer_3/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_3/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_3/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_3/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_3/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_3/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_3/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_3/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_3/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_3/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_3/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_3/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_3/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_3/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_3/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_3/output/add Fusing MatMul with Add for Node: bert/encoder/layer_4/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_4/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_4/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_4/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_4/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_4/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_4/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_4/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_4/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_4/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_4/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_4/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_4/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_4/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_4/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_4/output/add Fusing MatMul with Add for Node: bert/encoder/layer_5/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_5/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_5/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_5/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_5/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_5/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_5/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_5/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_5/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_5/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_5/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_5/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_5/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_5/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_5/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_5/output/add Fusing MatMul with Add for Node: bert/encoder/layer_6/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_6/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_6/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_6/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_6/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_6/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_6/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_6/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_6/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_6/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_6/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_6/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_6/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_6/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_6/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_6/output/add Fusing MatMul with Add for Node: bert/encoder/layer_7/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_7/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_7/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_7/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_7/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_7/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_7/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_7/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_7/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_7/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_7/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_7/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_7/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_7/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_7/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_7/output/add Fusing MatMul with Add for Node: bert/encoder/layer_8/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_8/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_8/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_8/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_8/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_8/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_8/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_8/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_8/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_8/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_8/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_8/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_8/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_8/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_8/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_8/output/add Fusing MatMul with Add for Node: bert/encoder/layer_9/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_9/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_9/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_9/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_9/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_9/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_9/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_9/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_9/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_9/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_9/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_9/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_9/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_9/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_9/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_9/output/add Fusing MatMul with Add for Node: bert/encoder/layer_10/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_10/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_10/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_10/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_10/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_10/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_10/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_10/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_10/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_10/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_10/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_10/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_10/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_10/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_10/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_10/output/add Fusing MatMul with Add for Node: bert/encoder/layer_11/attention/self/value/MatMul Skipping already processed Add Node: bert/encoder/layer_11/attention/self/value/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_11/attention/self/query/MatMul Skipping already processed Add Node: bert/encoder/layer_11/attention/self/query/BiasAdd Fusing MatMul with Add for Node: bert/encoder/layer_11/attention/self/key/MatMul Skipping already processed Add Node: bert/encoder/layer_11/attention/self/key/BiasAdd No Add node related to MatMul output: bert/encoder/layer_11/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_11/attention/self/MatMul_1. Executing regular MatMul. Fusing MatMul with 2Add for Node: bert/encoder/layer_11/attention/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_11/attention/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_11/attention/output/add Fusing MatMul with Add for Node: bert/encoder/layer_11/intermediate/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_11/intermediate/dense/BiasAdd Fusing MatMul with 2Add for Node: bert/encoder/layer_11/output/dense/MatMul Skipping already processed Add Node: bert/encoder/layer_11/output/dense/BiasAdd Skipping already processed Add Node: bert/encoder/layer_11/output/add Fusing MatMul with Add for Node: MatMul Skipping already processed Add Node: BiasAdd Node Execution Times: Node: unique_ids_graph_outputs_Identity__10, Execution Time: 0.000005 seconds Node: bert/encoder/Shape, Execution Time: 0.000004 seconds Node: bert/encoder/Shape__12, Execution Time: 0.000009 seconds Node: bert/encoder/strided_slice, Execution Time: 0.000062 seconds Node: bert/encoder/strided_slice__16, Execution Time: 0.000005 seconds Node: bert/encoder/strided_slice__17, Execution Time: 0.000005 seconds Node: bert/encoder/ones/packed_Unsqueeze__18, Execution Time: 0.000010 seconds Node: bert/encoder/ones/packed_Concat__21, Execution Time: 0.000010 seconds Node: bert/encoder/ones__22, Execution Time: 0.000004 seconds Node: bert/encoder/ones, Execution Time: 0.000011 seconds Node: bert/encoder/Reshape, Execution Time: 0.000006 seconds Node: bert/encoder/Cast, Execution Time: 0.000004 seconds Node: bert/encoder/mul, Execution Time: 0.028335 seconds Node: bert/encoder/layer_9/attention/self/ExpandDims, Execution Time: 0.000038 seconds Node: bert/encoder/layer_9/attention/self/sub, Execution Time: 0.006912 seconds Node: bert/encoder/layer_9/attention/self/mul_1, Execution Time: 0.000315 seconds Node: bert/embeddings/Reshape_2, Execution Time: 0.000020 seconds Node: bert/embeddings/Reshape, Execution Time: 0.000006 seconds Node: bert/embeddings/GatherV2, Execution Time: 0.000304 seconds Node: bert/embeddings/Reshape_1, Execution Time: 0.000008 seconds Node: bert/embeddings/one_hot, Execution Time: 0.000057 seconds Node: bert/embeddings/MatMul, Execution Time: 0.061418 seconds Node: bert/embeddings/Reshape_3, Execution Time: 0.000040 seconds Node: bert/embeddings/add, Execution Time: 0.002179 seconds Node: bert/embeddings/add_1, Execution Time: 0.001005 seconds Node: bert/embeddings/LayerNorm/moments/mean, Execution Time: 0.005431 seconds Node: bert/embeddings/LayerNorm/moments/SquaredDifference, Execution Time: 0.000726 seconds Node: bert/embeddings/LayerNorm/moments/SquaredDifference__72, Execution Time: 0.000863 seconds Node: bert/embeddings/LayerNorm/moments/variance, Execution Time: 0.000273 seconds Node: bert/embeddings/LayerNorm/batchnorm/add, Execution Time: 0.000082 seconds Node: bert/embeddings/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.010061 seconds Node: bert/embeddings/LayerNorm/batchnorm/Rsqrt__74, Execution Time: 0.005280 seconds Node: bert/embeddings/LayerNorm/batchnorm/mul, Execution Time: 0.000104 seconds Node: bert/embeddings/LayerNorm/batchnorm/mul_2, Execution Time: 0.000079 seconds Node: bert/embeddings/LayerNorm/batchnorm/sub, Execution Time: 0.000088 seconds Node: bert/embeddings/LayerNorm/batchnorm/mul_1, Execution Time: 0.000667 seconds Node: bert/embeddings/LayerNorm/batchnorm/add_1, Execution Time: 0.000691 seconds Node: bert/encoder/Reshape_1, Execution Time: 0.000034 seconds Matmul Fuse Node: bert/encoder/layer_0/attention/self/value/MatMul, Execution Time: 0.035522 seconds Node: bert/encoder/layer_0/attention/self/Reshape_2, Execution Time: 0.000023 seconds Node: bert/encoder/layer_0/attention/self/transpose_2, Execution Time: 0.000437 seconds Matmul Fuse Node: bert/encoder/layer_0/attention/self/query/MatMul, Execution Time: 0.003021 seconds Node: bert/encoder/layer_0/attention/self/Reshape, Execution Time: 0.000014 seconds Node: bert/encoder/layer_0/attention/self/transpose, Execution Time: 0.000255 seconds Matmul Fuse Node: bert/encoder/layer_0/attention/self/key/MatMul, Execution Time: 0.003390 seconds Node: bert/encoder/layer_0/attention/self/Reshape_1, Execution Time: 0.000015 seconds Node: bert/encoder/layer_0/attention/self/MatMul__306, Execution Time: 0.000262 seconds Node: bert/encoder/layer_0/attention/self/MatMul, Execution Time: 0.005664 seconds Node: bert/encoder/layer_0/attention/self/Mul, Execution Time: 0.001250 seconds Node: bert/encoder/layer_0/attention/self/add, Execution Time: 0.001806 seconds Node: bert/encoder/layer_0/attention/self/Softmax, Execution Time: 0.009676 seconds Node: bert/encoder/layer_0/attention/self/MatMul_1, Execution Time: 0.001447 seconds Node: bert/encoder/layer_0/attention/self/transpose_3, Execution Time: 0.000253 seconds Node: bert/encoder/layer_0/attention/self/Reshape_3, Execution Time: 0.000055 seconds Matmul Fuse Node: bert/encoder/layer_0/attention/output/dense/MatMul, Execution Time: 0.001492 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/moments/mean, Execution Time: 0.000218 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000390 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/moments/SquaredDifference__309, Execution Time: 0.000521 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/moments/variance, Execution Time: 0.000165 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000122 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000087 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/Rsqrt__311, Execution Time: 0.000131 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000120 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000082 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000095 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000568 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000690 seconds Matmul Fuse Node: bert/encoder/layer_0/intermediate/dense/MatMul, Execution Time: 0.006251 seconds Node: bert/encoder/layer_0/intermediate/dense/Pow, Execution Time: 0.017836 seconds Node: bert/encoder/layer_0/intermediate/dense/mul, Execution Time: 0.001201 seconds Node: bert/encoder/layer_0/intermediate/dense/add, Execution Time: 0.001495 seconds Node: bert/encoder/layer_0/intermediate/dense/mul_1, Execution Time: 0.001162 seconds Node: bert/encoder/layer_0/intermediate/dense/Tanh, Execution Time: 0.003510 seconds Node: bert/encoder/layer_0/intermediate/dense/add_1, Execution Time: 0.001209 seconds Node: bert/encoder/layer_0/intermediate/dense/mul_2, Execution Time: 0.001196 seconds Node: bert/encoder/layer_0/intermediate/dense/mul_3, Execution Time: 0.001443 seconds Matmul Fuse Node: bert/encoder/layer_0/output/dense/MatMul, Execution Time: 0.002903 seconds Node: bert/encoder/layer_0/output/LayerNorm/moments/mean, Execution Time: 0.000174 seconds Node: bert/encoder/layer_0/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000250 seconds Node: bert/encoder/layer_0/output/LayerNorm/moments/SquaredDifference__313, Execution Time: 0.000330 seconds Node: bert/encoder/layer_0/output/LayerNorm/moments/variance, Execution Time: 0.000158 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/add, Execution Time: 0.000081 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000059 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/Rsqrt__315, Execution Time: 0.000079 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/mul, Execution Time: 0.000078 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000091 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/sub, Execution Time: 0.000095 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000340 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000436 seconds Matmul Fuse Node: bert/encoder/layer_1/attention/self/value/MatMul, Execution Time: 0.002877 seconds Node: bert/encoder/layer_1/attention/self/Reshape_2, Execution Time: 0.000021 seconds Node: bert/encoder/layer_1/attention/self/transpose_2, Execution Time: 0.000252 seconds Matmul Fuse Node: bert/encoder/layer_1/attention/self/query/MatMul, Execution Time: 0.002599 seconds Node: bert/encoder/layer_1/attention/self/Reshape, Execution Time: 0.000014 seconds Node: bert/encoder/layer_1/attention/self/transpose, Execution Time: 0.000255 seconds Matmul Fuse Node: bert/encoder/layer_1/attention/self/key/MatMul, Execution Time: 0.002547 seconds Node: bert/encoder/layer_1/attention/self/Reshape_1, Execution Time: 0.000012 seconds Node: bert/encoder/layer_1/attention/self/MatMul__320, Execution Time: 0.000247 seconds Node: bert/encoder/layer_1/attention/self/MatMul, Execution Time: 0.001364 seconds Node: bert/encoder/layer_1/attention/self/Mul, Execution Time: 0.001176 seconds Node: bert/encoder/layer_1/attention/self/add, Execution Time: 0.001914 seconds Node: bert/encoder/layer_1/attention/self/Softmax, Execution Time: 0.001903 seconds Node: bert/encoder/layer_1/attention/self/MatMul_1, Execution Time: 0.001078 seconds Node: bert/encoder/layer_1/attention/self/transpose_3, Execution Time: 0.000241 seconds Node: bert/encoder/layer_1/attention/self/Reshape_3, Execution Time: 0.000048 seconds Matmul Fuse Node: bert/encoder/layer_1/attention/output/dense/MatMul, Execution Time: 0.001257 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/moments/mean, Execution Time: 0.000174 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000249 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/moments/SquaredDifference__323, Execution Time: 0.000343 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/moments/variance, Execution Time: 0.000155 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000082 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/Rsqrt__325, Execution Time: 0.000081 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000078 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000092 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000087 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000334 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000696 seconds Matmul Fuse Node: bert/encoder/layer_1/intermediate/dense/MatMul, Execution Time: 0.004094 seconds Node: bert/encoder/layer_1/intermediate/dense/Pow, Execution Time: 0.000729 seconds Node: bert/encoder/layer_1/intermediate/dense/mul, Execution Time: 0.001168 seconds Node: bert/encoder/layer_1/intermediate/dense/add, Execution Time: 0.001503 seconds Node: bert/encoder/layer_1/intermediate/dense/mul_1, Execution Time: 0.001165 seconds Node: bert/encoder/layer_1/intermediate/dense/Tanh, Execution Time: 0.001071 seconds Node: bert/encoder/layer_1/intermediate/dense/add_1, Execution Time: 0.001120 seconds Node: bert/encoder/layer_1/intermediate/dense/mul_2, Execution Time: 0.001135 seconds Node: bert/encoder/layer_1/intermediate/dense/mul_3, Execution Time: 0.001441 seconds Matmul Fuse Node: bert/encoder/layer_1/output/dense/MatMul, Execution Time: 0.002657 seconds Node: bert/encoder/layer_1/output/LayerNorm/moments/mean, Execution Time: 0.000176 seconds Node: bert/encoder/layer_1/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000243 seconds Node: bert/encoder/layer_1/output/LayerNorm/moments/SquaredDifference__327, Execution Time: 0.000338 seconds Node: bert/encoder/layer_1/output/LayerNorm/moments/variance, Execution Time: 0.000164 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/add, Execution Time: 0.000081 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/Rsqrt__329, Execution Time: 0.000079 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/mul, Execution Time: 0.000077 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000092 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/sub, Execution Time: 0.000083 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000336 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000424 seconds Matmul Fuse Node: bert/encoder/layer_2/attention/self/value/MatMul, Execution Time: 0.003581 seconds Node: bert/encoder/layer_2/attention/self/Reshape_2, Execution Time: 0.000026 seconds Node: bert/encoder/layer_2/attention/self/transpose_2, Execution Time: 0.000279 seconds Matmul Fuse Node: bert/encoder/layer_2/attention/self/query/MatMul, Execution Time: 0.003191 seconds Node: bert/encoder/layer_2/attention/self/Reshape, Execution Time: 0.000017 seconds Node: bert/encoder/layer_2/attention/self/transpose, Execution Time: 0.000271 seconds Matmul Fuse Node: bert/encoder/layer_2/attention/self/key/MatMul, Execution Time: 0.003162 seconds Node: bert/encoder/layer_2/attention/self/Reshape_1, Execution Time: 0.000015 seconds Node: bert/encoder/layer_2/attention/self/MatMul__334, Execution Time: 0.000279 seconds Node: bert/encoder/layer_2/attention/self/MatMul, Execution Time: 0.001417 seconds Node: bert/encoder/layer_2/attention/self/Mul, Execution Time: 0.001199 seconds Node: bert/encoder/layer_2/attention/self/add, Execution Time: 0.001837 seconds Node: bert/encoder/layer_2/attention/self/Softmax, Execution Time: 0.001921 seconds Node: bert/encoder/layer_2/attention/self/MatMul_1, Execution Time: 0.001089 seconds Node: bert/encoder/layer_2/attention/self/transpose_3, Execution Time: 0.000246 seconds Node: bert/encoder/layer_2/attention/self/Reshape_3, Execution Time: 0.000047 seconds Matmul Fuse Node: bert/encoder/layer_2/attention/output/dense/MatMul, Execution Time: 0.001225 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/moments/mean, Execution Time: 0.000175 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000243 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/moments/SquaredDifference__337, Execution Time: 0.000339 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/moments/variance, Execution Time: 0.000157 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000082 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000059 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/Rsqrt__339, Execution Time: 0.000095 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000081 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000096 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000080 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000582 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000878 seconds Matmul Fuse Node: bert/encoder/layer_2/intermediate/dense/MatMul, Execution Time: 0.004214 seconds Node: bert/encoder/layer_2/intermediate/dense/Pow, Execution Time: 0.000725 seconds Node: bert/encoder/layer_2/intermediate/dense/mul, Execution Time: 0.001163 seconds Node: bert/encoder/layer_2/intermediate/dense/add, Execution Time: 0.001479 seconds Node: bert/encoder/layer_2/intermediate/dense/mul_1, Execution Time: 0.001115 seconds Node: bert/encoder/layer_2/intermediate/dense/Tanh, Execution Time: 0.001115 seconds Node: bert/encoder/layer_2/intermediate/dense/add_1, Execution Time: 0.001124 seconds Node: bert/encoder/layer_2/intermediate/dense/mul_2, Execution Time: 0.001173 seconds Node: bert/encoder/layer_2/intermediate/dense/mul_3, Execution Time: 0.001371 seconds Matmul Fuse Node: bert/encoder/layer_2/output/dense/MatMul, Execution Time: 0.002838 seconds Node: bert/encoder/layer_2/output/LayerNorm/moments/mean, Execution Time: 0.000176 seconds Node: bert/encoder/layer_2/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000256 seconds Node: bert/encoder/layer_2/output/LayerNorm/moments/SquaredDifference__341, Execution Time: 0.000376 seconds Node: bert/encoder/layer_2/output/LayerNorm/moments/variance, Execution Time: 0.000160 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/add, Execution Time: 0.000082 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000059 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/Rsqrt__343, Execution Time: 0.000081 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/mul, Execution Time: 0.000089 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000085 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/sub, Execution Time: 0.000084 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000336 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000438 seconds Matmul Fuse Node: bert/encoder/layer_3/attention/self/value/MatMul, Execution Time: 0.002847 seconds Node: bert/encoder/layer_3/attention/self/Reshape_2, Execution Time: 0.000021 seconds Node: bert/encoder/layer_3/attention/self/transpose_2, Execution Time: 0.000258 seconds Matmul Fuse Node: bert/encoder/layer_3/attention/self/query/MatMul, Execution Time: 0.002538 seconds Node: bert/encoder/layer_3/attention/self/Reshape, Execution Time: 0.000013 seconds Node: bert/encoder/layer_3/attention/self/transpose, Execution Time: 0.000247 seconds Matmul Fuse Node: bert/encoder/layer_3/attention/self/key/MatMul, Execution Time: 0.002475 seconds Node: bert/encoder/layer_3/attention/self/Reshape_1, Execution Time: 0.000012 seconds Node: bert/encoder/layer_3/attention/self/MatMul__348, Execution Time: 0.000247 seconds Node: bert/encoder/layer_3/attention/self/MatMul, Execution Time: 0.001355 seconds Node: bert/encoder/layer_3/attention/self/Mul, Execution Time: 0.001203 seconds Node: bert/encoder/layer_3/attention/self/add, Execution Time: 0.001965 seconds Node: bert/encoder/layer_3/attention/self/Softmax, Execution Time: 0.002022 seconds Node: bert/encoder/layer_3/attention/self/MatMul_1, Execution Time: 0.001136 seconds Node: bert/encoder/layer_3/attention/self/transpose_3, Execution Time: 0.000241 seconds Node: bert/encoder/layer_3/attention/self/Reshape_3, Execution Time: 0.000049 seconds Matmul Fuse Node: bert/encoder/layer_3/attention/output/dense/MatMul, Execution Time: 0.001128 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/moments/mean, Execution Time: 0.000180 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000386 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/moments/SquaredDifference__351, Execution Time: 0.000455 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/moments/variance, Execution Time: 0.000154 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000088 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000059 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/Rsqrt__353, Execution Time: 0.000095 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000079 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000095 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000079 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000570 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000667 seconds Matmul Fuse Node: bert/encoder/layer_3/intermediate/dense/MatMul, Execution Time: 0.004129 seconds Node: bert/encoder/layer_3/intermediate/dense/Pow, Execution Time: 0.000726 seconds Node: bert/encoder/layer_3/intermediate/dense/mul, Execution Time: 0.001235 seconds Node: bert/encoder/layer_3/intermediate/dense/add, Execution Time: 0.001406 seconds Node: bert/encoder/layer_3/intermediate/dense/mul_1, Execution Time: 0.001123 seconds Node: bert/encoder/layer_3/intermediate/dense/Tanh, Execution Time: 0.001110 seconds Node: bert/encoder/layer_3/intermediate/dense/add_1, Execution Time: 0.001189 seconds Node: bert/encoder/layer_3/intermediate/dense/mul_2, Execution Time: 0.001317 seconds Node: bert/encoder/layer_3/intermediate/dense/mul_3, Execution Time: 0.001786 seconds Matmul Fuse Node: bert/encoder/layer_3/output/dense/MatMul, Execution Time: 0.003806 seconds Node: bert/encoder/layer_3/output/LayerNorm/moments/mean, Execution Time: 0.000238 seconds Node: bert/encoder/layer_3/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000318 seconds Node: bert/encoder/layer_3/output/LayerNorm/moments/SquaredDifference__355, Execution Time: 0.000446 seconds Node: bert/encoder/layer_3/output/LayerNorm/moments/variance, Execution Time: 0.000205 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/add, Execution Time: 0.000099 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000081 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/Rsqrt__357, Execution Time: 0.000107 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/mul, Execution Time: 0.000096 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000088 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/sub, Execution Time: 0.000113 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000434 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000534 seconds Matmul Fuse Node: bert/encoder/layer_4/attention/self/value/MatMul, Execution Time: 0.002899 seconds Node: bert/encoder/layer_4/attention/self/Reshape_2, Execution Time: 0.000021 seconds Node: bert/encoder/layer_4/attention/self/transpose_2, Execution Time: 0.000262 seconds Matmul Fuse Node: bert/encoder/layer_4/attention/self/query/MatMul, Execution Time: 0.002542 seconds Node: bert/encoder/layer_4/attention/self/Reshape, Execution Time: 0.000014 seconds Node: bert/encoder/layer_4/attention/self/transpose, Execution Time: 0.000249 seconds Matmul Fuse Node: bert/encoder/layer_4/attention/self/key/MatMul, Execution Time: 0.002438 seconds Node: bert/encoder/layer_4/attention/self/Reshape_1, Execution Time: 0.000012 seconds Node: bert/encoder/layer_4/attention/self/MatMul__362, Execution Time: 0.000249 seconds Node: bert/encoder/layer_4/attention/self/MatMul, Execution Time: 0.001404 seconds Node: bert/encoder/layer_4/attention/self/Mul, Execution Time: 0.001179 seconds Node: bert/encoder/layer_4/attention/self/add, Execution Time: 0.001856 seconds Node: bert/encoder/layer_4/attention/self/Softmax, Execution Time: 0.002107 seconds Node: bert/encoder/layer_4/attention/self/MatMul_1, Execution Time: 0.001110 seconds Node: bert/encoder/layer_4/attention/self/transpose_3, Execution Time: 0.000237 seconds Node: bert/encoder/layer_4/attention/self/Reshape_3, Execution Time: 0.000047 seconds Matmul Fuse Node: bert/encoder/layer_4/attention/output/dense/MatMul, Execution Time: 0.001309 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/moments/mean, Execution Time: 0.000175 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000243 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/moments/SquaredDifference__365, Execution Time: 0.000333 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/moments/variance, Execution Time: 0.000156 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000082 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000057 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/Rsqrt__367, Execution Time: 0.000079 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000079 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000095 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000078 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000587 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000684 seconds Matmul Fuse Node: bert/encoder/layer_4/intermediate/dense/MatMul, Execution Time: 0.004243 seconds Node: bert/encoder/layer_4/intermediate/dense/Pow, Execution Time: 0.000731 seconds Node: bert/encoder/layer_4/intermediate/dense/mul, Execution Time: 0.001199 seconds Node: bert/encoder/layer_4/intermediate/dense/add, Execution Time: 0.001486 seconds Node: bert/encoder/layer_4/intermediate/dense/mul_1, Execution Time: 0.001201 seconds Node: bert/encoder/layer_4/intermediate/dense/Tanh, Execution Time: 0.001122 seconds Node: bert/encoder/layer_4/intermediate/dense/add_1, Execution Time: 0.001201 seconds Node: bert/encoder/layer_4/intermediate/dense/mul_2, Execution Time: 0.001146 seconds Node: bert/encoder/layer_4/intermediate/dense/mul_3, Execution Time: 0.001469 seconds Matmul Fuse Node: bert/encoder/layer_4/output/dense/MatMul, Execution Time: 0.002865 seconds Node: bert/encoder/layer_4/output/LayerNorm/moments/mean, Execution Time: 0.000179 seconds Node: bert/encoder/layer_4/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000241 seconds Node: bert/encoder/layer_4/output/LayerNorm/moments/SquaredDifference__369, Execution Time: 0.000339 seconds Node: bert/encoder/layer_4/output/LayerNorm/moments/variance, Execution Time: 0.000156 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/add, Execution Time: 0.000081 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000059 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/Rsqrt__371, Execution Time: 0.000083 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/mul, Execution Time: 0.000079 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000092 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/sub, Execution Time: 0.000084 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000341 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000435 seconds Matmul Fuse Node: bert/encoder/layer_5/attention/self/value/MatMul, Execution Time: 0.003960 seconds Node: bert/encoder/layer_5/attention/self/Reshape_2, Execution Time: 0.000022 seconds Node: bert/encoder/layer_5/attention/self/transpose_2, Execution Time: 0.000257 seconds Matmul Fuse Node: bert/encoder/layer_5/attention/self/query/MatMul, Execution Time: 0.002440 seconds Node: bert/encoder/layer_5/attention/self/Reshape, Execution Time: 0.000013 seconds Node: bert/encoder/layer_5/attention/self/transpose, Execution Time: 0.000252 seconds Matmul Fuse Node: bert/encoder/layer_5/attention/self/key/MatMul, Execution Time: 0.002446 seconds Node: bert/encoder/layer_5/attention/self/Reshape_1, Execution Time: 0.000011 seconds Node: bert/encoder/layer_5/attention/self/MatMul__376, Execution Time: 0.000251 seconds Node: bert/encoder/layer_5/attention/self/MatMul, Execution Time: 0.001244 seconds Node: bert/encoder/layer_5/attention/self/Mul, Execution Time: 0.001164 seconds Node: bert/encoder/layer_5/attention/self/add, Execution Time: 0.001820 seconds Node: bert/encoder/layer_5/attention/self/Softmax, Execution Time: 0.001953 seconds Node: bert/encoder/layer_5/attention/self/MatMul_1, Execution Time: 0.001064 seconds Node: bert/encoder/layer_5/attention/self/transpose_3, Execution Time: 0.000276 seconds Node: bert/encoder/layer_5/attention/self/Reshape_3, Execution Time: 0.000045 seconds Matmul Fuse Node: bert/encoder/layer_5/attention/output/dense/MatMul, Execution Time: 0.001237 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/moments/mean, Execution Time: 0.000190 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000249 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/moments/SquaredDifference__379, Execution Time: 0.000471 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/moments/variance, Execution Time: 0.000163 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000084 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000059 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/Rsqrt__381, Execution Time: 0.000080 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000081 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000098 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000080 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000573 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000682 seconds Matmul Fuse Node: bert/encoder/layer_5/intermediate/dense/MatMul, Execution Time: 0.004223 seconds Node: bert/encoder/layer_5/intermediate/dense/Pow, Execution Time: 0.000738 seconds Node: bert/encoder/layer_5/intermediate/dense/mul, Execution Time: 0.001173 seconds Node: bert/encoder/layer_5/intermediate/dense/add, Execution Time: 0.001558 seconds Node: bert/encoder/layer_5/intermediate/dense/mul_1, Execution Time: 0.001120 seconds Node: bert/encoder/layer_5/intermediate/dense/Tanh, Execution Time: 0.001077 seconds Node: bert/encoder/layer_5/intermediate/dense/add_1, Execution Time: 0.001116 seconds Node: bert/encoder/layer_5/intermediate/dense/mul_2, Execution Time: 0.001096 seconds Node: bert/encoder/layer_5/intermediate/dense/mul_3, Execution Time: 0.001424 seconds Matmul Fuse Node: bert/encoder/layer_5/output/dense/MatMul, Execution Time: 0.002838 seconds Node: bert/encoder/layer_5/output/LayerNorm/moments/mean, Execution Time: 0.000184 seconds Node: bert/encoder/layer_5/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000249 seconds Node: bert/encoder/layer_5/output/LayerNorm/moments/SquaredDifference__383, Execution Time: 0.000327 seconds Node: bert/encoder/layer_5/output/LayerNorm/moments/variance, Execution Time: 0.000155 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/add, Execution Time: 0.000081 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/Rsqrt__385, Execution Time: 0.000090 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/mul, Execution Time: 0.000082 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000075 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/sub, Execution Time: 0.000085 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000334 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000436 seconds Matmul Fuse Node: bert/encoder/layer_6/attention/self/value/MatMul, Execution Time: 0.002932 seconds Node: bert/encoder/layer_6/attention/self/Reshape_2, Execution Time: 0.000021 seconds Node: bert/encoder/layer_6/attention/self/transpose_2, Execution Time: 0.000256 seconds Matmul Fuse Node: bert/encoder/layer_6/attention/self/query/MatMul, Execution Time: 0.002506 seconds Node: bert/encoder/layer_6/attention/self/Reshape, Execution Time: 0.000014 seconds Node: bert/encoder/layer_6/attention/self/transpose, Execution Time: 0.000253 seconds Matmul Fuse Node: bert/encoder/layer_6/attention/self/key/MatMul, Execution Time: 0.002383 seconds Node: bert/encoder/layer_6/attention/self/Reshape_1, Execution Time: 0.000013 seconds Node: bert/encoder/layer_6/attention/self/MatMul__390, Execution Time: 0.000243 seconds Node: bert/encoder/layer_6/attention/self/MatMul, Execution Time: 0.001354 seconds Node: bert/encoder/layer_6/attention/self/Mul, Execution Time: 0.001219 seconds Node: bert/encoder/layer_6/attention/self/add, Execution Time: 0.001827 seconds Node: bert/encoder/layer_6/attention/self/Softmax, Execution Time: 0.002038 seconds Node: bert/encoder/layer_6/attention/self/MatMul_1, Execution Time: 0.001150 seconds Node: bert/encoder/layer_6/attention/self/transpose_3, Execution Time: 0.000246 seconds Node: bert/encoder/layer_6/attention/self/Reshape_3, Execution Time: 0.000048 seconds Matmul Fuse Node: bert/encoder/layer_6/attention/output/dense/MatMul, Execution Time: 0.001310 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/moments/mean, Execution Time: 0.000202 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000269 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/moments/SquaredDifference__393, Execution Time: 0.000342 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/moments/variance, Execution Time: 0.000169 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000093 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000066 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/Rsqrt__395, Execution Time: 0.000086 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000098 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000085 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000093 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000468 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000721 seconds Matmul Fuse Node: bert/encoder/layer_6/intermediate/dense/MatMul, Execution Time: 0.004496 seconds Node: bert/encoder/layer_6/intermediate/dense/Pow, Execution Time: 0.000743 seconds Node: bert/encoder/layer_6/intermediate/dense/mul, Execution Time: 0.001149 seconds Node: bert/encoder/layer_6/intermediate/dense/add, Execution Time: 0.001630 seconds Node: bert/encoder/layer_6/intermediate/dense/mul_1, Execution Time: 0.001597 seconds Node: bert/encoder/layer_6/intermediate/dense/Tanh, Execution Time: 0.001440 seconds Node: bert/encoder/layer_6/intermediate/dense/add_1, Execution Time: 0.001472 seconds Node: bert/encoder/layer_6/intermediate/dense/mul_2, Execution Time: 0.001459 seconds Node: bert/encoder/layer_6/intermediate/dense/mul_3, Execution Time: 0.001819 seconds Matmul Fuse Node: bert/encoder/layer_6/output/dense/MatMul, Execution Time: 0.002854 seconds Node: bert/encoder/layer_6/output/LayerNorm/moments/mean, Execution Time: 0.000175 seconds Node: bert/encoder/layer_6/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000244 seconds Node: bert/encoder/layer_6/output/LayerNorm/moments/SquaredDifference__397, Execution Time: 0.000333 seconds Node: bert/encoder/layer_6/output/LayerNorm/moments/variance, Execution Time: 0.000155 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/add, Execution Time: 0.000081 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000057 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/Rsqrt__399, Execution Time: 0.000099 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/mul, Execution Time: 0.000079 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000077 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/sub, Execution Time: 0.000078 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000335 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000436 seconds Matmul Fuse Node: bert/encoder/layer_7/attention/self/value/MatMul, Execution Time: 0.002808 seconds Node: bert/encoder/layer_7/attention/self/Reshape_2, Execution Time: 0.000021 seconds Node: bert/encoder/layer_7/attention/self/transpose_2, Execution Time: 0.000246 seconds Matmul Fuse Node: bert/encoder/layer_7/attention/self/query/MatMul, Execution Time: 0.002673 seconds Node: bert/encoder/layer_7/attention/self/Reshape, Execution Time: 0.000015 seconds Node: bert/encoder/layer_7/attention/self/transpose, Execution Time: 0.000253 seconds Matmul Fuse Node: bert/encoder/layer_7/attention/self/key/MatMul, Execution Time: 0.002464 seconds Node: bert/encoder/layer_7/attention/self/Reshape_1, Execution Time: 0.000012 seconds Node: bert/encoder/layer_7/attention/self/MatMul__404, Execution Time: 0.000244 seconds Node: bert/encoder/layer_7/attention/self/MatMul, Execution Time: 0.001390 seconds Node: bert/encoder/layer_7/attention/self/Mul, Execution Time: 0.001194 seconds Node: bert/encoder/layer_7/attention/self/add, Execution Time: 0.001980 seconds Node: bert/encoder/layer_7/attention/self/Softmax, Execution Time: 0.001891 seconds Node: bert/encoder/layer_7/attention/self/MatMul_1, Execution Time: 0.001181 seconds Node: bert/encoder/layer_7/attention/self/transpose_3, Execution Time: 0.000238 seconds Node: bert/encoder/layer_7/attention/self/Reshape_3, Execution Time: 0.000046 seconds Matmul Fuse Node: bert/encoder/layer_7/attention/output/dense/MatMul, Execution Time: 0.001251 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/moments/mean, Execution Time: 0.000179 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000251 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/moments/SquaredDifference__407, Execution Time: 0.000348 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/moments/variance, Execution Time: 0.000175 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000082 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000058 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/Rsqrt__409, Execution Time: 0.000095 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000082 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000098 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000087 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000552 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000713 seconds Matmul Fuse Node: bert/encoder/layer_7/intermediate/dense/MatMul, Execution Time: 0.004113 seconds Node: bert/encoder/layer_7/intermediate/dense/Pow, Execution Time: 0.000735 seconds Node: bert/encoder/layer_7/intermediate/dense/mul, Execution Time: 0.001155 seconds Node: bert/encoder/layer_7/intermediate/dense/add, Execution Time: 0.001512 seconds Node: bert/encoder/layer_7/intermediate/dense/mul_1, Execution Time: 0.001128 seconds Node: bert/encoder/layer_7/intermediate/dense/Tanh, Execution Time: 0.001093 seconds Node: bert/encoder/layer_7/intermediate/dense/add_1, Execution Time: 0.001129 seconds Node: bert/encoder/layer_7/intermediate/dense/mul_2, Execution Time: 0.001164 seconds Node: bert/encoder/layer_7/intermediate/dense/mul_3, Execution Time: 0.001491 seconds Matmul Fuse Node: bert/encoder/layer_7/output/dense/MatMul, Execution Time: 0.002885 seconds Node: bert/encoder/layer_7/output/LayerNorm/moments/mean, Execution Time: 0.000183 seconds Node: bert/encoder/layer_7/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000250 seconds Node: bert/encoder/layer_7/output/LayerNorm/moments/SquaredDifference__411, Execution Time: 0.000327 seconds Node: bert/encoder/layer_7/output/LayerNorm/moments/variance, Execution Time: 0.000154 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/add, Execution Time: 0.000082 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000058 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/Rsqrt__413, Execution Time: 0.000085 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/mul, Execution Time: 0.000085 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000082 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/sub, Execution Time: 0.000079 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000333 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000437 seconds Matmul Fuse Node: bert/encoder/layer_8/attention/self/value/MatMul, Execution Time: 0.002903 seconds Node: bert/encoder/layer_8/attention/self/Reshape_2, Execution Time: 0.000022 seconds Node: bert/encoder/layer_8/attention/self/transpose_2, Execution Time: 0.000252 seconds Matmul Fuse Node: bert/encoder/layer_8/attention/self/query/MatMul, Execution Time: 0.002493 seconds Node: bert/encoder/layer_8/attention/self/Reshape, Execution Time: 0.000014 seconds Node: bert/encoder/layer_8/attention/self/transpose, Execution Time: 0.000250 seconds Matmul Fuse Node: bert/encoder/layer_8/attention/self/key/MatMul, Execution Time: 0.002375 seconds Node: bert/encoder/layer_8/attention/self/Reshape_1, Execution Time: 0.000012 seconds Node: bert/encoder/layer_8/attention/self/MatMul__418, Execution Time: 0.000246 seconds Node: bert/encoder/layer_8/attention/self/MatMul, Execution Time: 0.001337 seconds Node: bert/encoder/layer_8/attention/self/Mul, Execution Time: 0.001168 seconds Node: bert/encoder/layer_8/attention/self/add, Execution Time: 0.001858 seconds Node: bert/encoder/layer_8/attention/self/Softmax, Execution Time: 0.002032 seconds Node: bert/encoder/layer_8/attention/self/MatMul_1, Execution Time: 0.001093 seconds Node: bert/encoder/layer_8/attention/self/transpose_3, Execution Time: 0.000242 seconds Node: bert/encoder/layer_8/attention/self/Reshape_3, Execution Time: 0.000047 seconds Matmul Fuse Node: bert/encoder/layer_8/attention/output/dense/MatMul, Execution Time: 0.001178 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/moments/mean, Execution Time: 0.000181 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000245 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/moments/SquaredDifference__421, Execution Time: 0.000330 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/moments/variance, Execution Time: 0.000159 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000083 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000058 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/Rsqrt__423, Execution Time: 0.000095 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000082 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000082 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000078 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000559 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000686 seconds Matmul Fuse Node: bert/encoder/layer_8/intermediate/dense/MatMul, Execution Time: 0.004353 seconds Node: bert/encoder/layer_8/intermediate/dense/Pow, Execution Time: 0.000730 seconds Node: bert/encoder/layer_8/intermediate/dense/mul, Execution Time: 0.001183 seconds Node: bert/encoder/layer_8/intermediate/dense/add, Execution Time: 0.001496 seconds Node: bert/encoder/layer_8/intermediate/dense/mul_1, Execution Time: 0.001133 seconds Node: bert/encoder/layer_8/intermediate/dense/Tanh, Execution Time: 0.001168 seconds Node: bert/encoder/layer_8/intermediate/dense/add_1, Execution Time: 0.001154 seconds Node: bert/encoder/layer_8/intermediate/dense/mul_2, Execution Time: 0.001138 seconds Node: bert/encoder/layer_8/intermediate/dense/mul_3, Execution Time: 0.001387 seconds Matmul Fuse Node: bert/encoder/layer_8/output/dense/MatMul, Execution Time: 0.002991 seconds Node: bert/encoder/layer_8/output/LayerNorm/moments/mean, Execution Time: 0.000182 seconds Node: bert/encoder/layer_8/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000258 seconds Node: bert/encoder/layer_8/output/LayerNorm/moments/SquaredDifference__425, Execution Time: 0.000333 seconds Node: bert/encoder/layer_8/output/LayerNorm/moments/variance, Execution Time: 0.000160 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/add, Execution Time: 0.000081 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000059 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/Rsqrt__427, Execution Time: 0.000095 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/mul, Execution Time: 0.000082 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000095 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/sub, Execution Time: 0.000082 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000342 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000425 seconds Matmul Fuse Node: bert/encoder/layer_9/attention/self/value/MatMul, Execution Time: 0.002894 seconds Node: bert/encoder/layer_9/attention/self/Reshape_2, Execution Time: 0.000022 seconds Node: bert/encoder/layer_9/attention/self/transpose_2, Execution Time: 0.000255 seconds Matmul Fuse Node: bert/encoder/layer_9/attention/self/query/MatMul, Execution Time: 0.002460 seconds Node: bert/encoder/layer_9/attention/self/Reshape, Execution Time: 0.000013 seconds Node: bert/encoder/layer_9/attention/self/transpose, Execution Time: 0.000249 seconds Matmul Fuse Node: bert/encoder/layer_9/attention/self/key/MatMul, Execution Time: 0.002440 seconds Node: bert/encoder/layer_9/attention/self/Reshape_1, Execution Time: 0.000012 seconds Node: bert/encoder/layer_9/attention/self/MatMul__432, Execution Time: 0.000247 seconds Node: bert/encoder/layer_9/attention/self/MatMul, Execution Time: 0.001341 seconds Node: bert/encoder/layer_9/attention/self/Mul, Execution Time: 0.001163 seconds Node: bert/encoder/layer_9/attention/self/add, Execution Time: 0.001809 seconds Node: bert/encoder/layer_9/attention/self/Softmax, Execution Time: 0.001965 seconds Node: bert/encoder/layer_9/attention/self/MatMul_1, Execution Time: 0.001070 seconds Node: bert/encoder/layer_9/attention/self/transpose_3, Execution Time: 0.000237 seconds Node: bert/encoder/layer_9/attention/self/Reshape_3, Execution Time: 0.000045 seconds Matmul Fuse Node: bert/encoder/layer_9/attention/output/dense/MatMul, Execution Time: 0.001157 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/moments/mean, Execution Time: 0.000197 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000245 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/moments/SquaredDifference__435, Execution Time: 0.000330 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/moments/variance, Execution Time: 0.000159 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000082 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000059 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/Rsqrt__437, Execution Time: 0.000084 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000101 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000084 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000078 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000559 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000698 seconds Matmul Fuse Node: bert/encoder/layer_9/intermediate/dense/MatMul, Execution Time: 0.004072 seconds Node: bert/encoder/layer_9/intermediate/dense/Pow, Execution Time: 0.000727 seconds Node: bert/encoder/layer_9/intermediate/dense/mul, Execution Time: 0.001199 seconds Node: bert/encoder/layer_9/intermediate/dense/add, Execution Time: 0.001454 seconds Node: bert/encoder/layer_9/intermediate/dense/mul_1, Execution Time: 0.001218 seconds Node: bert/encoder/layer_9/intermediate/dense/Tanh, Execution Time: 0.001086 seconds Node: bert/encoder/layer_9/intermediate/dense/add_1, Execution Time: 0.001137 seconds Node: bert/encoder/layer_9/intermediate/dense/mul_2, Execution Time: 0.001107 seconds Node: bert/encoder/layer_9/intermediate/dense/mul_3, Execution Time: 0.001379 seconds Matmul Fuse Node: bert/encoder/layer_9/output/dense/MatMul, Execution Time: 0.002824 seconds Node: bert/encoder/layer_9/output/LayerNorm/moments/mean, Execution Time: 0.000182 seconds Node: bert/encoder/layer_9/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000251 seconds Node: bert/encoder/layer_9/output/LayerNorm/moments/SquaredDifference__439, Execution Time: 0.000329 seconds Node: bert/encoder/layer_9/output/LayerNorm/moments/variance, Execution Time: 0.000154 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/add, Execution Time: 0.000083 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000059 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/Rsqrt__441, Execution Time: 0.000084 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/mul, Execution Time: 0.000081 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000096 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/sub, Execution Time: 0.000082 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000331 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000436 seconds Matmul Fuse Node: bert/encoder/layer_10/attention/self/value/MatMul, Execution Time: 0.004091 seconds Node: bert/encoder/layer_10/attention/self/Reshape_2, Execution Time: 0.000021 seconds Node: bert/encoder/layer_10/attention/self/transpose_2, Execution Time: 0.000257 seconds Matmul Fuse Node: bert/encoder/layer_10/attention/self/query/MatMul, Execution Time: 0.002522 seconds Node: bert/encoder/layer_10/attention/self/Reshape, Execution Time: 0.000013 seconds Node: bert/encoder/layer_10/attention/self/transpose, Execution Time: 0.000260 seconds Matmul Fuse Node: bert/encoder/layer_10/attention/self/key/MatMul, Execution Time: 0.002380 seconds Node: bert/encoder/layer_10/attention/self/Reshape_1, Execution Time: 0.000012 seconds Node: bert/encoder/layer_10/attention/self/MatMul__446, Execution Time: 0.000246 seconds Node: bert/encoder/layer_10/attention/self/MatMul, Execution Time: 0.001328 seconds Node: bert/encoder/layer_10/attention/self/Mul, Execution Time: 0.001195 seconds Node: bert/encoder/layer_10/attention/self/add, Execution Time: 0.001870 seconds Node: bert/encoder/layer_10/attention/self/Softmax, Execution Time: 0.001986 seconds Node: bert/encoder/layer_10/attention/self/MatMul_1, Execution Time: 0.001072 seconds Node: bert/encoder/layer_10/attention/self/transpose_3, Execution Time: 0.000238 seconds Node: bert/encoder/layer_10/attention/self/Reshape_3, Execution Time: 0.000051 seconds Matmul Fuse Node: bert/encoder/layer_10/attention/output/dense/MatMul, Execution Time: 0.001109 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/moments/mean, Execution Time: 0.000175 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000308 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/moments/SquaredDifference__449, Execution Time: 0.000433 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/moments/variance, Execution Time: 0.000211 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000099 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000074 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/Rsqrt__451, Execution Time: 0.000107 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000101 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000102 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000097 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000704 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000876 seconds Matmul Fuse Node: bert/encoder/layer_10/intermediate/dense/MatMul, Execution Time: 0.005988 seconds Node: bert/encoder/layer_10/intermediate/dense/Pow, Execution Time: 0.000705 seconds Node: bert/encoder/layer_10/intermediate/dense/mul, Execution Time: 0.001194 seconds Node: bert/encoder/layer_10/intermediate/dense/add, Execution Time: 0.001401 seconds Node: bert/encoder/layer_10/intermediate/dense/mul_1, Execution Time: 0.001104 seconds Node: bert/encoder/layer_10/intermediate/dense/Tanh, Execution Time: 0.001094 seconds Node: bert/encoder/layer_10/intermediate/dense/add_1, Execution Time: 0.001137 seconds Node: bert/encoder/layer_10/intermediate/dense/mul_2, Execution Time: 0.001155 seconds Node: bert/encoder/layer_10/intermediate/dense/mul_3, Execution Time: 0.001385 seconds Matmul Fuse Node: bert/encoder/layer_10/output/dense/MatMul, Execution Time: 0.002557 seconds Node: bert/encoder/layer_10/output/LayerNorm/moments/mean, Execution Time: 0.000184 seconds Node: bert/encoder/layer_10/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000247 seconds Node: bert/encoder/layer_10/output/LayerNorm/moments/SquaredDifference__453, Execution Time: 0.000327 seconds Node: bert/encoder/layer_10/output/LayerNorm/moments/variance, Execution Time: 0.000155 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/add, Execution Time: 0.000081 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000059 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/Rsqrt__455, Execution Time: 0.000105 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/mul, Execution Time: 0.000086 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000094 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/sub, Execution Time: 0.000080 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000334 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000441 seconds Matmul Fuse Node: bert/encoder/layer_11/attention/self/value/MatMul, Execution Time: 0.002807 seconds Node: bert/encoder/layer_11/attention/self/Reshape_2, Execution Time: 0.000021 seconds Node: bert/encoder/layer_11/attention/self/transpose_2, Execution Time: 0.000258 seconds Matmul Fuse Node: bert/encoder/layer_11/attention/self/query/MatMul, Execution Time: 0.002520 seconds Node: bert/encoder/layer_11/attention/self/Reshape, Execution Time: 0.000013 seconds Node: bert/encoder/layer_11/attention/self/transpose, Execution Time: 0.000254 seconds Matmul Fuse Node: bert/encoder/layer_11/attention/self/key/MatMul, Execution Time: 0.002469 seconds Node: bert/encoder/layer_11/attention/self/Reshape_1, Execution Time: 0.000012 seconds Node: bert/encoder/layer_11/attention/self/MatMul__460, Execution Time: 0.000247 seconds Node: bert/encoder/layer_11/attention/self/MatMul, Execution Time: 0.001390 seconds Node: bert/encoder/layer_11/attention/self/Mul, Execution Time: 0.001185 seconds Node: bert/encoder/layer_11/attention/self/add, Execution Time: 0.001868 seconds Node: bert/encoder/layer_11/attention/self/Softmax, Execution Time: 0.001983 seconds Node: bert/encoder/layer_11/attention/self/MatMul_1, Execution Time: 0.001104 seconds Node: bert/encoder/layer_11/attention/self/transpose_3, Execution Time: 0.000240 seconds Node: bert/encoder/layer_11/attention/self/Reshape_3, Execution Time: 0.000048 seconds Matmul Fuse Node: bert/encoder/layer_11/attention/output/dense/MatMul, Execution Time: 0.001165 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/moments/mean, Execution Time: 0.000172 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000242 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/moments/SquaredDifference__463, Execution Time: 0.000333 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/moments/variance, Execution Time: 0.000166 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000079 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000058 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/Rsqrt__465, Execution Time: 0.000095 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000076 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000074 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000076 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000562 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000669 seconds Matmul Fuse Node: bert/encoder/layer_11/intermediate/dense/MatMul, Execution Time: 0.004179 seconds Node: bert/encoder/layer_11/intermediate/dense/Pow, Execution Time: 0.000729 seconds Node: bert/encoder/layer_11/intermediate/dense/mul, Execution Time: 0.001209 seconds Node: bert/encoder/layer_11/intermediate/dense/add, Execution Time: 0.001439 seconds Node: bert/encoder/layer_11/intermediate/dense/mul_1, Execution Time: 0.001104 seconds Node: bert/encoder/layer_11/intermediate/dense/Tanh, Execution Time: 0.001094 seconds Node: bert/encoder/layer_11/intermediate/dense/add_1, Execution Time: 0.001142 seconds Node: bert/encoder/layer_11/intermediate/dense/mul_2, Execution Time: 0.001143 seconds Node: bert/encoder/layer_11/intermediate/dense/mul_3, Execution Time: 0.001404 seconds Matmul Fuse Node: bert/encoder/layer_11/output/dense/MatMul, Execution Time: 0.002955 seconds Node: bert/encoder/layer_11/output/LayerNorm/moments/mean, Execution Time: 0.000190 seconds Node: bert/encoder/layer_11/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000246 seconds Node: bert/encoder/layer_11/output/LayerNorm/moments/SquaredDifference__467, Execution Time: 0.000330 seconds Node: bert/encoder/layer_11/output/LayerNorm/moments/variance, Execution Time: 0.000155 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/add, Execution Time: 0.000082 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000058 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/Rsqrt__469, Execution Time: 0.000086 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/mul, Execution Time: 0.000084 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000095 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/sub, Execution Time: 0.000080 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000335 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000436 seconds Matmul Fuse Node: MatMul, Execution Time: 0.004485 seconds Node: Reshape_1, Execution Time: 0.000021 seconds Node: transpose, Execution Time: 0.000088 seconds Node: unstack, Execution Time: 0.000050 seconds Node: unstack__490, Execution Time: 0.000005 seconds Node: unstack__488, Execution Time: 0.000006 seconds Total Execution Time: 0.676711 seconds Total Matmul Fuse Execution Time: 0.241226 seconds Execution complete. Total execution time: 0.680221 seconds Model outputs: {'unstack:1': array([nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan], dtype=float16), 'unstack:0': array(None, dtype=object), 'unique_ids:0': array([], dtype=int64)} Execution order: ['unique_ids_graph_outputs_Identity__10', 'bert/encoder/Shape', 'bert/encoder/Shape__12', 'bert/encoder/strided_slice', 'bert/encoder/strided_slice__16', 'bert/encoder/strided_slice__17', 'bert/encoder/ones/packed_Unsqueeze__18', 'bert/encoder/ones/packed_Concat__21', 'bert/encoder/ones__22', 'bert/encoder/ones', 'bert/encoder/Reshape', 'bert/encoder/Cast', 'bert/encoder/mul', 'bert/encoder/layer_9/attention/self/ExpandDims', 'bert/encoder/layer_9/attention/self/sub', 'bert/encoder/layer_9/attention/self/mul_1', 'bert/embeddings/Reshape_2', 'bert/embeddings/Reshape', 'bert/embeddings/GatherV2', 'bert/embeddings/Reshape_1', 'bert/embeddings/one_hot', 'bert/embeddings/MatMul', 'bert/embeddings/Reshape_3', 'bert/embeddings/add', 'bert/embeddings/add_1', 'bert/embeddings/LayerNorm/moments/mean', 'bert/embeddings/LayerNorm/moments/SquaredDifference', 'bert/embeddings/LayerNorm/moments/SquaredDifference__72', 'bert/embeddings/LayerNorm/moments/variance', 'bert/embeddings/LayerNorm/batchnorm/add', 'bert/embeddings/LayerNorm/batchnorm/Rsqrt', 'bert/embeddings/LayerNorm/batchnorm/Rsqrt__74', 'bert/embeddings/LayerNorm/batchnorm/mul', 'bert/embeddings/LayerNorm/batchnorm/mul_2', 'bert/embeddings/LayerNorm/batchnorm/sub', 'bert/embeddings/LayerNorm/batchnorm/mul_1', 'bert/embeddings/LayerNorm/batchnorm/add_1', 'bert/encoder/Reshape_1', 'bert/encoder/layer_0/attention/self/value/MatMul', 'bert/encoder/layer_0/attention/self/value/BiasAdd', 'bert/encoder/layer_0/attention/self/Reshape_2', 'bert/encoder/layer_0/attention/self/transpose_2', 'bert/encoder/layer_0/attention/self/query/MatMul', 'bert/encoder/layer_0/attention/self/query/BiasAdd', 'bert/encoder/layer_0/attention/self/Reshape', 'bert/encoder/layer_0/attention/self/transpose', 'bert/encoder/layer_0/attention/self/key/MatMul', 'bert/encoder/layer_0/attention/self/key/BiasAdd', 'bert/encoder/layer_0/attention/self/Reshape_1', 'bert/encoder/layer_0/attention/self/MatMul__306', 'bert/encoder/layer_0/attention/self/MatMul', 'bert/encoder/layer_0/attention/self/Mul', 'bert/encoder/layer_0/attention/self/add', 'bert/encoder/layer_0/attention/self/Softmax', 'bert/encoder/layer_0/attention/self/MatMul_1', 'bert/encoder/layer_0/attention/self/transpose_3', 'bert/encoder/layer_0/attention/self/Reshape_3', 'bert/encoder/layer_0/attention/output/dense/MatMul', 'bert/encoder/layer_0/attention/output/dense/BiasAdd', 'bert/encoder/layer_0/attention/output/add', 'bert/encoder/layer_0/attention/output/LayerNorm/moments/mean', 'bert/encoder/layer_0/attention/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_0/attention/output/LayerNorm/moments/SquaredDifference__309', 'bert/encoder/layer_0/attention/output/LayerNorm/moments/variance', 'bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/Rsqrt__311', 'bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/add_1', 'bert/encoder/layer_0/intermediate/dense/MatMul', 'bert/encoder/layer_0/intermediate/dense/BiasAdd', 'bert/encoder/layer_0/intermediate/dense/Pow', 'bert/encoder/layer_0/intermediate/dense/mul', 'bert/encoder/layer_0/intermediate/dense/add', 'bert/encoder/layer_0/intermediate/dense/mul_1', 'bert/encoder/layer_0/intermediate/dense/Tanh', 'bert/encoder/layer_0/intermediate/dense/add_1', 'bert/encoder/layer_0/intermediate/dense/mul_2', 'bert/encoder/layer_0/intermediate/dense/mul_3', 'bert/encoder/layer_0/output/dense/MatMul', 'bert/encoder/layer_0/output/dense/BiasAdd', 'bert/encoder/layer_0/output/add', 'bert/encoder/layer_0/output/LayerNorm/moments/mean', 'bert/encoder/layer_0/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_0/output/LayerNorm/moments/SquaredDifference__313', 'bert/encoder/layer_0/output/LayerNorm/moments/variance', 'bert/encoder/layer_0/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_0/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_0/output/LayerNorm/batchnorm/Rsqrt__315', 'bert/encoder/layer_0/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_0/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_0/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_0/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_0/output/LayerNorm/batchnorm/add_1', 'bert/encoder/layer_1/attention/self/value/MatMul', 'bert/encoder/layer_1/attention/self/value/BiasAdd', 'bert/encoder/layer_1/attention/self/Reshape_2', 'bert/encoder/layer_1/attention/self/transpose_2', 'bert/encoder/layer_1/attention/self/query/MatMul', 'bert/encoder/layer_1/attention/self/query/BiasAdd', 'bert/encoder/layer_1/attention/self/Reshape', 'bert/encoder/layer_1/attention/self/transpose', 'bert/encoder/layer_1/attention/self/key/MatMul', 'bert/encoder/layer_1/attention/self/key/BiasAdd', 'bert/encoder/layer_1/attention/self/Reshape_1', 'bert/encoder/layer_1/attention/self/MatMul__320', 'bert/encoder/layer_1/attention/self/MatMul', 'bert/encoder/layer_1/attention/self/Mul', 'bert/encoder/layer_1/attention/self/add', 'bert/encoder/layer_1/attention/self/Softmax', 'bert/encoder/layer_1/attention/self/MatMul_1', 'bert/encoder/layer_1/attention/self/transpose_3', 'bert/encoder/layer_1/attention/self/Reshape_3', 'bert/encoder/layer_1/attention/output/dense/MatMul', 'bert/encoder/layer_1/attention/output/dense/BiasAdd', 'bert/encoder/layer_1/attention/output/add', 'bert/encoder/layer_1/attention/output/LayerNorm/moments/mean', 'bert/encoder/layer_1/attention/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_1/attention/output/LayerNorm/moments/SquaredDifference__323', 'bert/encoder/layer_1/attention/output/LayerNorm/moments/variance', 'bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/Rsqrt__325', 'bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/add_1', 'bert/encoder/layer_1/intermediate/dense/MatMul', 'bert/encoder/layer_1/intermediate/dense/BiasAdd', 'bert/encoder/layer_1/intermediate/dense/Pow', 'bert/encoder/layer_1/intermediate/dense/mul', 'bert/encoder/layer_1/intermediate/dense/add', 'bert/encoder/layer_1/intermediate/dense/mul_1', 'bert/encoder/layer_1/intermediate/dense/Tanh', 'bert/encoder/layer_1/intermediate/dense/add_1', 'bert/encoder/layer_1/intermediate/dense/mul_2', 'bert/encoder/layer_1/intermediate/dense/mul_3', 'bert/encoder/layer_1/output/dense/MatMul', 'bert/encoder/layer_1/output/dense/BiasAdd', 'bert/encoder/layer_1/output/add', 'bert/encoder/layer_1/output/LayerNorm/moments/mean', 'bert/encoder/layer_1/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_1/output/LayerNorm/moments/SquaredDifference__327', 'bert/encoder/layer_1/output/LayerNorm/moments/variance', 'bert/encoder/layer_1/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_1/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_1/output/LayerNorm/batchnorm/Rsqrt__329', 'bert/encoder/layer_1/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_1/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_1/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_1/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_1/output/LayerNorm/batchnorm/add_1', 'bert/encoder/layer_2/attention/self/value/MatMul', 'bert/encoder/layer_2/attention/self/value/BiasAdd', 'bert/encoder/layer_2/attention/self/Reshape_2', 'bert/encoder/layer_2/attention/self/transpose_2', 'bert/encoder/layer_2/attention/self/query/MatMul', 'bert/encoder/layer_2/attention/self/query/BiasAdd', 'bert/encoder/layer_2/attention/self/Reshape', 'bert/encoder/layer_2/attention/self/transpose', 'bert/encoder/layer_2/attention/self/key/MatMul', 'bert/encoder/layer_2/attention/self/key/BiasAdd', 'bert/encoder/layer_2/attention/self/Reshape_1', 'bert/encoder/layer_2/attention/self/MatMul__334', 'bert/encoder/layer_2/attention/self/MatMul', 'bert/encoder/layer_2/attention/self/Mul', 'bert/encoder/layer_2/attention/self/add', 'bert/encoder/layer_2/attention/self/Softmax', 'bert/encoder/layer_2/attention/self/MatMul_1', 'bert/encoder/layer_2/attention/self/transpose_3', 'bert/encoder/layer_2/attention/self/Reshape_3', 'bert/encoder/layer_2/attention/output/dense/MatMul', 'bert/encoder/layer_2/attention/output/dense/BiasAdd', 'bert/encoder/layer_2/attention/output/add', 'bert/encoder/layer_2/attention/output/LayerNorm/moments/mean', 'bert/encoder/layer_2/attention/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_2/attention/output/LayerNorm/moments/SquaredDifference__337', 'bert/encoder/layer_2/attention/output/LayerNorm/moments/variance', 'bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/Rsqrt__339', 'bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/add_1', 'bert/encoder/layer_2/intermediate/dense/MatMul', 'bert/encoder/layer_2/intermediate/dense/BiasAdd', 'bert/encoder/layer_2/intermediate/dense/Pow', 'bert/encoder/layer_2/intermediate/dense/mul', 'bert/encoder/layer_2/intermediate/dense/add', 'bert/encoder/layer_2/intermediate/dense/mul_1', 'bert/encoder/layer_2/intermediate/dense/Tanh', 'bert/encoder/layer_2/intermediate/dense/add_1', 'bert/encoder/layer_2/intermediate/dense/mul_2', 'bert/encoder/layer_2/intermediate/dense/mul_3', 'bert/encoder/layer_2/output/dense/MatMul', 'bert/encoder/layer_2/output/dense/BiasAdd', 'bert/encoder/layer_2/output/add', 'bert/encoder/layer_2/output/LayerNorm/moments/mean', 'bert/encoder/layer_2/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_2/output/LayerNorm/moments/SquaredDifference__341', 'bert/encoder/layer_2/output/LayerNorm/moments/variance', 'bert/encoder/layer_2/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_2/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_2/output/LayerNorm/batchnorm/Rsqrt__343', 'bert/encoder/layer_2/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_2/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_2/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_2/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_2/output/LayerNorm/batchnorm/add_1', 'bert/encoder/layer_3/attention/self/value/MatMul', 'bert/encoder/layer_3/attention/self/value/BiasAdd', 'bert/encoder/layer_3/attention/self/Reshape_2', 'bert/encoder/layer_3/attention/self/transpose_2', 'bert/encoder/layer_3/attention/self/query/MatMul', 'bert/encoder/layer_3/attention/self/query/BiasAdd', 'bert/encoder/layer_3/attention/self/Reshape', 'bert/encoder/layer_3/attention/self/transpose', 'bert/encoder/layer_3/attention/self/key/MatMul', 'bert/encoder/layer_3/attention/self/key/BiasAdd', 'bert/encoder/layer_3/attention/self/Reshape_1', 'bert/encoder/layer_3/attention/self/MatMul__348', 'bert/encoder/layer_3/attention/self/MatMul', 'bert/encoder/layer_3/attention/self/Mul', 'bert/encoder/layer_3/attention/self/add', 'bert/encoder/layer_3/attention/self/Softmax', 'bert/encoder/layer_3/attention/self/MatMul_1', 'bert/encoder/layer_3/attention/self/transpose_3', 'bert/encoder/layer_3/attention/self/Reshape_3', 'bert/encoder/layer_3/attention/output/dense/MatMul', 'bert/encoder/layer_3/attention/output/dense/BiasAdd', 'bert/encoder/layer_3/attention/output/add', 'bert/encoder/layer_3/attention/output/LayerNorm/moments/mean', 'bert/encoder/layer_3/attention/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_3/attention/output/LayerNorm/moments/SquaredDifference__351', 'bert/encoder/layer_3/attention/output/LayerNorm/moments/variance', 'bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/Rsqrt__353', 'bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/add_1', 'bert/encoder/layer_3/intermediate/dense/MatMul', 'bert/encoder/layer_3/intermediate/dense/BiasAdd', 'bert/encoder/layer_3/intermediate/dense/Pow', 'bert/encoder/layer_3/intermediate/dense/mul', 'bert/encoder/layer_3/intermediate/dense/add', 'bert/encoder/layer_3/intermediate/dense/mul_1', 'bert/encoder/layer_3/intermediate/dense/Tanh', 'bert/encoder/layer_3/intermediate/dense/add_1', 'bert/encoder/layer_3/intermediate/dense/mul_2', 'bert/encoder/layer_3/intermediate/dense/mul_3', 'bert/encoder/layer_3/output/dense/MatMul', 'bert/encoder/layer_3/output/dense/BiasAdd', 'bert/encoder/layer_3/output/add', 'bert/encoder/layer_3/output/LayerNorm/moments/mean', 'bert/encoder/layer_3/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_3/output/LayerNorm/moments/SquaredDifference__355', 'bert/encoder/layer_3/output/LayerNorm/moments/variance', 'bert/encoder/layer_3/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_3/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_3/output/LayerNorm/batchnorm/Rsqrt__357', 'bert/encoder/layer_3/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_3/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_3/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_3/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_3/output/LayerNorm/batchnorm/add_1', 'bert/encoder/layer_4/attention/self/value/MatMul', 'bert/encoder/layer_4/attention/self/value/BiasAdd', 'bert/encoder/layer_4/attention/self/Reshape_2', 'bert/encoder/layer_4/attention/self/transpose_2', 'bert/encoder/layer_4/attention/self/query/MatMul', 'bert/encoder/layer_4/attention/self/query/BiasAdd', 'bert/encoder/layer_4/attention/self/Reshape', 'bert/encoder/layer_4/attention/self/transpose', 'bert/encoder/layer_4/attention/self/key/MatMul', 'bert/encoder/layer_4/attention/self/key/BiasAdd', 'bert/encoder/layer_4/attention/self/Reshape_1', 'bert/encoder/layer_4/attention/self/MatMul__362', 'bert/encoder/layer_4/attention/self/MatMul', 'bert/encoder/layer_4/attention/self/Mul', 'bert/encoder/layer_4/attention/self/add', 'bert/encoder/layer_4/attention/self/Softmax', 'bert/encoder/layer_4/attention/self/MatMul_1', 'bert/encoder/layer_4/attention/self/transpose_3', 'bert/encoder/layer_4/attention/self/Reshape_3', 'bert/encoder/layer_4/attention/output/dense/MatMul', 'bert/encoder/layer_4/attention/output/dense/BiasAdd', 'bert/encoder/layer_4/attention/output/add', 'bert/encoder/layer_4/attention/output/LayerNorm/moments/mean', 'bert/encoder/layer_4/attention/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_4/attention/output/LayerNorm/moments/SquaredDifference__365', 'bert/encoder/layer_4/attention/output/LayerNorm/moments/variance', 'bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/Rsqrt__367', 'bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/add_1', 'bert/encoder/layer_4/intermediate/dense/MatMul', 'bert/encoder/layer_4/intermediate/dense/BiasAdd', 'bert/encoder/layer_4/intermediate/dense/Pow', 'bert/encoder/layer_4/intermediate/dense/mul', 'bert/encoder/layer_4/intermediate/dense/add', 'bert/encoder/layer_4/intermediate/dense/mul_1', 'bert/encoder/layer_4/intermediate/dense/Tanh', 'bert/encoder/layer_4/intermediate/dense/add_1', 'bert/encoder/layer_4/intermediate/dense/mul_2', 'bert/encoder/layer_4/intermediate/dense/mul_3', 'bert/encoder/layer_4/output/dense/MatMul', 'bert/encoder/layer_4/output/dense/BiasAdd', 'bert/encoder/layer_4/output/add', 'bert/encoder/layer_4/output/LayerNorm/moments/mean', 'bert/encoder/layer_4/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_4/output/LayerNorm/moments/SquaredDifference__369', 'bert/encoder/layer_4/output/LayerNorm/moments/variance', 'bert/encoder/layer_4/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_4/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_4/output/LayerNorm/batchnorm/Rsqrt__371', 'bert/encoder/layer_4/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_4/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_4/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_4/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_4/output/LayerNorm/batchnorm/add_1', 'bert/encoder/layer_5/attention/self/value/MatMul', 'bert/encoder/layer_5/attention/self/value/BiasAdd', 'bert/encoder/layer_5/attention/self/Reshape_2', 'bert/encoder/layer_5/attention/self/transpose_2', 'bert/encoder/layer_5/attention/self/query/MatMul', 'bert/encoder/layer_5/attention/self/query/BiasAdd', 'bert/encoder/layer_5/attention/self/Reshape', 'bert/encoder/layer_5/attention/self/transpose', 'bert/encoder/layer_5/attention/self/key/MatMul', 'bert/encoder/layer_5/attention/self/key/BiasAdd', 'bert/encoder/layer_5/attention/self/Reshape_1', 'bert/encoder/layer_5/attention/self/MatMul__376', 'bert/encoder/layer_5/attention/self/MatMul', 'bert/encoder/layer_5/attention/self/Mul', 'bert/encoder/layer_5/attention/self/add', 'bert/encoder/layer_5/attention/self/Softmax', 'bert/encoder/layer_5/attention/self/MatMul_1', 'bert/encoder/layer_5/attention/self/transpose_3', 'bert/encoder/layer_5/attention/self/Reshape_3', 'bert/encoder/layer_5/attention/output/dense/MatMul', 'bert/encoder/layer_5/attention/output/dense/BiasAdd', 'bert/encoder/layer_5/attention/output/add', 'bert/encoder/layer_5/attention/output/LayerNorm/moments/mean', 'bert/encoder/layer_5/attention/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_5/attention/output/LayerNorm/moments/SquaredDifference__379', 'bert/encoder/layer_5/attention/output/LayerNorm/moments/variance', 'bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/Rsqrt__381', 'bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/add_1', 'bert/encoder/layer_5/intermediate/dense/MatMul', 'bert/encoder/layer_5/intermediate/dense/BiasAdd', 'bert/encoder/layer_5/intermediate/dense/Pow', 'bert/encoder/layer_5/intermediate/dense/mul', 'bert/encoder/layer_5/intermediate/dense/add', 'bert/encoder/layer_5/intermediate/dense/mul_1', 'bert/encoder/layer_5/intermediate/dense/Tanh', 'bert/encoder/layer_5/intermediate/dense/add_1', 'bert/encoder/layer_5/intermediate/dense/mul_2', 'bert/encoder/layer_5/intermediate/dense/mul_3', 'bert/encoder/layer_5/output/dense/MatMul', 'bert/encoder/layer_5/output/dense/BiasAdd', 'bert/encoder/layer_5/output/add', 'bert/encoder/layer_5/output/LayerNorm/moments/mean', 'bert/encoder/layer_5/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_5/output/LayerNorm/moments/SquaredDifference__383', 'bert/encoder/layer_5/output/LayerNorm/moments/variance', 'bert/encoder/layer_5/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_5/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_5/output/LayerNorm/batchnorm/Rsqrt__385', 'bert/encoder/layer_5/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_5/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_5/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_5/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_5/output/LayerNorm/batchnorm/add_1', 'bert/encoder/layer_6/attention/self/value/MatMul', 'bert/encoder/layer_6/attention/self/value/BiasAdd', 'bert/encoder/layer_6/attention/self/Reshape_2', 'bert/encoder/layer_6/attention/self/transpose_2', 'bert/encoder/layer_6/attention/self/query/MatMul', 'bert/encoder/layer_6/attention/self/query/BiasAdd', 'bert/encoder/layer_6/attention/self/Reshape', 'bert/encoder/layer_6/attention/self/transpose', 'bert/encoder/layer_6/attention/self/key/MatMul', 'bert/encoder/layer_6/attention/self/key/BiasAdd', 'bert/encoder/layer_6/attention/self/Reshape_1', 'bert/encoder/layer_6/attention/self/MatMul__390', 'bert/encoder/layer_6/attention/self/MatMul', 'bert/encoder/layer_6/attention/self/Mul', 'bert/encoder/layer_6/attention/self/add', 'bert/encoder/layer_6/attention/self/Softmax', 'bert/encoder/layer_6/attention/self/MatMul_1', 'bert/encoder/layer_6/attention/self/transpose_3', 'bert/encoder/layer_6/attention/self/Reshape_3', 'bert/encoder/layer_6/attention/output/dense/MatMul', 'bert/encoder/layer_6/attention/output/dense/BiasAdd', 'bert/encoder/layer_6/attention/output/add', 'bert/encoder/layer_6/attention/output/LayerNorm/moments/mean', 'bert/encoder/layer_6/attention/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_6/attention/output/LayerNorm/moments/SquaredDifference__393', 'bert/encoder/layer_6/attention/output/LayerNorm/moments/variance', 'bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/Rsqrt__395', 'bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/add_1', 'bert/encoder/layer_6/intermediate/dense/MatMul', 'bert/encoder/layer_6/intermediate/dense/BiasAdd', 'bert/encoder/layer_6/intermediate/dense/Pow', 'bert/encoder/layer_6/intermediate/dense/mul', 'bert/encoder/layer_6/intermediate/dense/add', 'bert/encoder/layer_6/intermediate/dense/mul_1', 'bert/encoder/layer_6/intermediate/dense/Tanh', 'bert/encoder/layer_6/intermediate/dense/add_1', 'bert/encoder/layer_6/intermediate/dense/mul_2', 'bert/encoder/layer_6/intermediate/dense/mul_3', 'bert/encoder/layer_6/output/dense/MatMul', 'bert/encoder/layer_6/output/dense/BiasAdd', 'bert/encoder/layer_6/output/add', 'bert/encoder/layer_6/output/LayerNorm/moments/mean', 'bert/encoder/layer_6/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_6/output/LayerNorm/moments/SquaredDifference__397', 'bert/encoder/layer_6/output/LayerNorm/moments/variance', 'bert/encoder/layer_6/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_6/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_6/output/LayerNorm/batchnorm/Rsqrt__399', 'bert/encoder/layer_6/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_6/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_6/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_6/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_6/output/LayerNorm/batchnorm/add_1', 'bert/encoder/layer_7/attention/self/value/MatMul', 'bert/encoder/layer_7/attention/self/value/BiasAdd', 'bert/encoder/layer_7/attention/self/Reshape_2', 'bert/encoder/layer_7/attention/self/transpose_2', 'bert/encoder/layer_7/attention/self/query/MatMul', 'bert/encoder/layer_7/attention/self/query/BiasAdd', 'bert/encoder/layer_7/attention/self/Reshape', 'bert/encoder/layer_7/attention/self/transpose', 'bert/encoder/layer_7/attention/self/key/MatMul', 'bert/encoder/layer_7/attention/self/key/BiasAdd', 'bert/encoder/layer_7/attention/self/Reshape_1', 'bert/encoder/layer_7/attention/self/MatMul__404', 'bert/encoder/layer_7/attention/self/MatMul', 'bert/encoder/layer_7/attention/self/Mul', 'bert/encoder/layer_7/attention/self/add', 'bert/encoder/layer_7/attention/self/Softmax', 'bert/encoder/layer_7/attention/self/MatMul_1', 'bert/encoder/layer_7/attention/self/transpose_3', 'bert/encoder/layer_7/attention/self/Reshape_3', 'bert/encoder/layer_7/attention/output/dense/MatMul', 'bert/encoder/layer_7/attention/output/dense/BiasAdd', 'bert/encoder/layer_7/attention/output/add', 'bert/encoder/layer_7/attention/output/LayerNorm/moments/mean', 'bert/encoder/layer_7/attention/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_7/attention/output/LayerNorm/moments/SquaredDifference__407', 'bert/encoder/layer_7/attention/output/LayerNorm/moments/variance', 'bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/Rsqrt__409', 'bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/add_1', 'bert/encoder/layer_7/intermediate/dense/MatMul', 'bert/encoder/layer_7/intermediate/dense/BiasAdd', 'bert/encoder/layer_7/intermediate/dense/Pow', 'bert/encoder/layer_7/intermediate/dense/mul', 'bert/encoder/layer_7/intermediate/dense/add', 'bert/encoder/layer_7/intermediate/dense/mul_1', 'bert/encoder/layer_7/intermediate/dense/Tanh', 'bert/encoder/layer_7/intermediate/dense/add_1', 'bert/encoder/layer_7/intermediate/dense/mul_2', 'bert/encoder/layer_7/intermediate/dense/mul_3', 'bert/encoder/layer_7/output/dense/MatMul', 'bert/encoder/layer_7/output/dense/BiasAdd', 'bert/encoder/layer_7/output/add', 'bert/encoder/layer_7/output/LayerNorm/moments/mean', 'bert/encoder/layer_7/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_7/output/LayerNorm/moments/SquaredDifference__411', 'bert/encoder/layer_7/output/LayerNorm/moments/variance', 'bert/encoder/layer_7/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_7/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_7/output/LayerNorm/batchnorm/Rsqrt__413', 'bert/encoder/layer_7/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_7/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_7/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_7/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_7/output/LayerNorm/batchnorm/add_1', 'bert/encoder/layer_8/attention/self/value/MatMul', 'bert/encoder/layer_8/attention/self/value/BiasAdd', 'bert/encoder/layer_8/attention/self/Reshape_2', 'bert/encoder/layer_8/attention/self/transpose_2', 'bert/encoder/layer_8/attention/self/query/MatMul', 'bert/encoder/layer_8/attention/self/query/BiasAdd', 'bert/encoder/layer_8/attention/self/Reshape', 'bert/encoder/layer_8/attention/self/transpose', 'bert/encoder/layer_8/attention/self/key/MatMul', 'bert/encoder/layer_8/attention/self/key/BiasAdd', 'bert/encoder/layer_8/attention/self/Reshape_1', 'bert/encoder/layer_8/attention/self/MatMul__418', 'bert/encoder/layer_8/attention/self/MatMul', 'bert/encoder/layer_8/attention/self/Mul', 'bert/encoder/layer_8/attention/self/add', 'bert/encoder/layer_8/attention/self/Softmax', 'bert/encoder/layer_8/attention/self/MatMul_1', 'bert/encoder/layer_8/attention/self/transpose_3', 'bert/encoder/layer_8/attention/self/Reshape_3', 'bert/encoder/layer_8/attention/output/dense/MatMul', 'bert/encoder/layer_8/attention/output/dense/BiasAdd', 'bert/encoder/layer_8/attention/output/add', 'bert/encoder/layer_8/attention/output/LayerNorm/moments/mean', 'bert/encoder/layer_8/attention/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_8/attention/output/LayerNorm/moments/SquaredDifference__421', 'bert/encoder/layer_8/attention/output/LayerNorm/moments/variance', 'bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/Rsqrt__423', 'bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/add_1', 'bert/encoder/layer_8/intermediate/dense/MatMul', 'bert/encoder/layer_8/intermediate/dense/BiasAdd', 'bert/encoder/layer_8/intermediate/dense/Pow', 'bert/encoder/layer_8/intermediate/dense/mul', 'bert/encoder/layer_8/intermediate/dense/add', 'bert/encoder/layer_8/intermediate/dense/mul_1', 'bert/encoder/layer_8/intermediate/dense/Tanh', 'bert/encoder/layer_8/intermediate/dense/add_1', 'bert/encoder/layer_8/intermediate/dense/mul_2', 'bert/encoder/layer_8/intermediate/dense/mul_3', 'bert/encoder/layer_8/output/dense/MatMul', 'bert/encoder/layer_8/output/dense/BiasAdd', 'bert/encoder/layer_8/output/add', 'bert/encoder/layer_8/output/LayerNorm/moments/mean', 'bert/encoder/layer_8/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_8/output/LayerNorm/moments/SquaredDifference__425', 'bert/encoder/layer_8/output/LayerNorm/moments/variance', 'bert/encoder/layer_8/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_8/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_8/output/LayerNorm/batchnorm/Rsqrt__427', 'bert/encoder/layer_8/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_8/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_8/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_8/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_8/output/LayerNorm/batchnorm/add_1', 'bert/encoder/layer_9/attention/self/value/MatMul', 'bert/encoder/layer_9/attention/self/value/BiasAdd', 'bert/encoder/layer_9/attention/self/Reshape_2', 'bert/encoder/layer_9/attention/self/transpose_2', 'bert/encoder/layer_9/attention/self/query/MatMul', 'bert/encoder/layer_9/attention/self/query/BiasAdd', 'bert/encoder/layer_9/attention/self/Reshape', 'bert/encoder/layer_9/attention/self/transpose', 'bert/encoder/layer_9/attention/self/key/MatMul', 'bert/encoder/layer_9/attention/self/key/BiasAdd', 'bert/encoder/layer_9/attention/self/Reshape_1', 'bert/encoder/layer_9/attention/self/MatMul__432', 'bert/encoder/layer_9/attention/self/MatMul', 'bert/encoder/layer_9/attention/self/Mul', 'bert/encoder/layer_9/attention/self/add', 'bert/encoder/layer_9/attention/self/Softmax', 'bert/encoder/layer_9/attention/self/MatMul_1', 'bert/encoder/layer_9/attention/self/transpose_3', 'bert/encoder/layer_9/attention/self/Reshape_3', 'bert/encoder/layer_9/attention/output/dense/MatMul', 'bert/encoder/layer_9/attention/output/dense/BiasAdd', 'bert/encoder/layer_9/attention/output/add', 'bert/encoder/layer_9/attention/output/LayerNorm/moments/mean', 'bert/encoder/layer_9/attention/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_9/attention/output/LayerNorm/moments/SquaredDifference__435', 'bert/encoder/layer_9/attention/output/LayerNorm/moments/variance', 'bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/Rsqrt__437', 'bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/add_1', 'bert/encoder/layer_9/intermediate/dense/MatMul', 'bert/encoder/layer_9/intermediate/dense/BiasAdd', 'bert/encoder/layer_9/intermediate/dense/Pow', 'bert/encoder/layer_9/intermediate/dense/mul', 'bert/encoder/layer_9/intermediate/dense/add', 'bert/encoder/layer_9/intermediate/dense/mul_1', 'bert/encoder/layer_9/intermediate/dense/Tanh', 'bert/encoder/layer_9/intermediate/dense/add_1', 'bert/encoder/layer_9/intermediate/dense/mul_2', 'bert/encoder/layer_9/intermediate/dense/mul_3', 'bert/encoder/layer_9/output/dense/MatMul', 'bert/encoder/layer_9/output/dense/BiasAdd', 'bert/encoder/layer_9/output/add', 'bert/encoder/layer_9/output/LayerNorm/moments/mean', 'bert/encoder/layer_9/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_9/output/LayerNorm/moments/SquaredDifference__439', 'bert/encoder/layer_9/output/LayerNorm/moments/variance', 'bert/encoder/layer_9/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_9/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_9/output/LayerNorm/batchnorm/Rsqrt__441', 'bert/encoder/layer_9/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_9/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_9/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_9/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_9/output/LayerNorm/batchnorm/add_1', 'bert/encoder/layer_10/attention/self/value/MatMul', 'bert/encoder/layer_10/attention/self/value/BiasAdd', 'bert/encoder/layer_10/attention/self/Reshape_2', 'bert/encoder/layer_10/attention/self/transpose_2', 'bert/encoder/layer_10/attention/self/query/MatMul', 'bert/encoder/layer_10/attention/self/query/BiasAdd', 'bert/encoder/layer_10/attention/self/Reshape', 'bert/encoder/layer_10/attention/self/transpose', 'bert/encoder/layer_10/attention/self/key/MatMul', 'bert/encoder/layer_10/attention/self/key/BiasAdd', 'bert/encoder/layer_10/attention/self/Reshape_1', 'bert/encoder/layer_10/attention/self/MatMul__446', 'bert/encoder/layer_10/attention/self/MatMul', 'bert/encoder/layer_10/attention/self/Mul', 'bert/encoder/layer_10/attention/self/add', 'bert/encoder/layer_10/attention/self/Softmax', 'bert/encoder/layer_10/attention/self/MatMul_1', 'bert/encoder/layer_10/attention/self/transpose_3', 'bert/encoder/layer_10/attention/self/Reshape_3', 'bert/encoder/layer_10/attention/output/dense/MatMul', 'bert/encoder/layer_10/attention/output/dense/BiasAdd', 'bert/encoder/layer_10/attention/output/add', 'bert/encoder/layer_10/attention/output/LayerNorm/moments/mean', 'bert/encoder/layer_10/attention/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_10/attention/output/LayerNorm/moments/SquaredDifference__449', 'bert/encoder/layer_10/attention/output/LayerNorm/moments/variance', 'bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/Rsqrt__451', 'bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/add_1', 'bert/encoder/layer_10/intermediate/dense/MatMul', 'bert/encoder/layer_10/intermediate/dense/BiasAdd', 'bert/encoder/layer_10/intermediate/dense/Pow', 'bert/encoder/layer_10/intermediate/dense/mul', 'bert/encoder/layer_10/intermediate/dense/add', 'bert/encoder/layer_10/intermediate/dense/mul_1', 'bert/encoder/layer_10/intermediate/dense/Tanh', 'bert/encoder/layer_10/intermediate/dense/add_1', 'bert/encoder/layer_10/intermediate/dense/mul_2', 'bert/encoder/layer_10/intermediate/dense/mul_3', 'bert/encoder/layer_10/output/dense/MatMul', 'bert/encoder/layer_10/output/dense/BiasAdd', 'bert/encoder/layer_10/output/add', 'bert/encoder/layer_10/output/LayerNorm/moments/mean', 'bert/encoder/layer_10/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_10/output/LayerNorm/moments/SquaredDifference__453', 'bert/encoder/layer_10/output/LayerNorm/moments/variance', 'bert/encoder/layer_10/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_10/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_10/output/LayerNorm/batchnorm/Rsqrt__455', 'bert/encoder/layer_10/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_10/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_10/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_10/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_10/output/LayerNorm/batchnorm/add_1', 'bert/encoder/layer_11/attention/self/value/MatMul', 'bert/encoder/layer_11/attention/self/value/BiasAdd', 'bert/encoder/layer_11/attention/self/Reshape_2', 'bert/encoder/layer_11/attention/self/transpose_2', 'bert/encoder/layer_11/attention/self/query/MatMul', 'bert/encoder/layer_11/attention/self/query/BiasAdd', 'bert/encoder/layer_11/attention/self/Reshape', 'bert/encoder/layer_11/attention/self/transpose', 'bert/encoder/layer_11/attention/self/key/MatMul', 'bert/encoder/layer_11/attention/self/key/BiasAdd', 'bert/encoder/layer_11/attention/self/Reshape_1', 'bert/encoder/layer_11/attention/self/MatMul__460', 'bert/encoder/layer_11/attention/self/MatMul', 'bert/encoder/layer_11/attention/self/Mul', 'bert/encoder/layer_11/attention/self/add', 'bert/encoder/layer_11/attention/self/Softmax', 'bert/encoder/layer_11/attention/self/MatMul_1', 'bert/encoder/layer_11/attention/self/transpose_3', 'bert/encoder/layer_11/attention/self/Reshape_3', 'bert/encoder/layer_11/attention/output/dense/MatMul', 'bert/encoder/layer_11/attention/output/dense/BiasAdd', 'bert/encoder/layer_11/attention/output/add', 'bert/encoder/layer_11/attention/output/LayerNorm/moments/mean', 'bert/encoder/layer_11/attention/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_11/attention/output/LayerNorm/moments/SquaredDifference__463', 'bert/encoder/layer_11/attention/output/LayerNorm/moments/variance', 'bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/Rsqrt__465', 'bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/add_1', 'bert/encoder/layer_11/intermediate/dense/MatMul', 'bert/encoder/layer_11/intermediate/dense/BiasAdd', 'bert/encoder/layer_11/intermediate/dense/Pow', 'bert/encoder/layer_11/intermediate/dense/mul', 'bert/encoder/layer_11/intermediate/dense/add', 'bert/encoder/layer_11/intermediate/dense/mul_1', 'bert/encoder/layer_11/intermediate/dense/Tanh', 'bert/encoder/layer_11/intermediate/dense/add_1', 'bert/encoder/layer_11/intermediate/dense/mul_2', 'bert/encoder/layer_11/intermediate/dense/mul_3', 'bert/encoder/layer_11/output/dense/MatMul', 'bert/encoder/layer_11/output/dense/BiasAdd', 'bert/encoder/layer_11/output/add', 'bert/encoder/layer_11/output/LayerNorm/moments/mean', 'bert/encoder/layer_11/output/LayerNorm/moments/SquaredDifference', 'bert/encoder/layer_11/output/LayerNorm/moments/SquaredDifference__467', 'bert/encoder/layer_11/output/LayerNorm/moments/variance', 'bert/encoder/layer_11/output/LayerNorm/batchnorm/add', 'bert/encoder/layer_11/output/LayerNorm/batchnorm/Rsqrt', 'bert/encoder/layer_11/output/LayerNorm/batchnorm/Rsqrt__469', 'bert/encoder/layer_11/output/LayerNorm/batchnorm/mul', 'bert/encoder/layer_11/output/LayerNorm/batchnorm/mul_2', 'bert/encoder/layer_11/output/LayerNorm/batchnorm/sub', 'bert/encoder/layer_11/output/LayerNorm/batchnorm/mul_1', 'bert/encoder/layer_11/output/LayerNorm/batchnorm/add_1', 'MatMul', 'BiasAdd', 'Reshape_1', 'transpose', 'unstack', 'unstack__490', 'unstack__488'] Generating '/tmp/nsys-report-ae10.qdstrm' [1/8] [0% ] nsys-report-0123.nsys-rep [1/8] [0% ] nsys-report-0123.nsys-rep [1/8] [5% ] nsys-report-0123.nsys-rep [1/8] [6% ] nsys-report-0123.nsys-rep [1/8] [11% ] nsys-report-0123.nsys-rep [1/8] [9% ] nsys-report-0123.nsys-rep [1/8] [8% ] nsys-report-0123.nsys-rep [1/8] [6% ] nsys-report-0123.nsys-rep [1/8] [====25% ] nsys-report-0123.nsys-rep [1/8] [===22% ] nsys-report-0123.nsys-rep [1/8] [==20% ] nsys-report-0123.nsys-rep [1/8] [==18% ] nsys-report-0123.nsys-rep [1/8] [=17% ] nsys-report-0123.nsys-rep [1/8] [==19% ] nsys-report-0123.nsys-rep [1/8] [==20% ] nsys-report-0123.nsys-rep [1/8] [==21% ] nsys-report-0123.nsys-rep [1/8] [===22% ] nsys-report-0123.nsys-rep [1/8] [===23% ] nsys-report-0123.nsys-rep [1/8] [===24% ] nsys-report-0123.nsys-rep [1/8] [====25% ] nsys-report-0123.nsys-rep [1/8] [====26% ] nsys-report-0123.nsys-rep [1/8] [====28% ] nsys-report-0123.nsys-rep [1/8] [=====30% ] nsys-report-0123.nsys-rep [1/8] [======34% ] nsys-report-0123.nsys-rep [1/8] [=======38% ] nsys-report-0123.nsys-rep [1/8] [==========48% ] nsys-report-0123.nsys-rep [1/8] [============57% ] nsys-report-0123.nsys-rep [1/8] [===============67% ] nsys-report-0123.nsys-rep [1/8] [================71% ] nsys-report-0123.nsys-rep [1/8] [=================72% ] nsys-report-0123.nsys-rep [1/8] [==================76% ] nsys-report-0123.nsys-rep [1/8] [====================83% ] nsys-report-0123.nsys-rep [1/8] [====================84% ] nsys-report-0123.nsys-rep [1/8] [====================85% ] nsys-report-0123.nsys-rep [1/8] [=====================86% ] nsys-report-0123.nsys-rep [1/8] [=====================87% ] nsys-report-0123.nsys-rep [1/8] [=====================88% ] nsys-report-0123.nsys-rep [1/8] [========================97% ] nsys-report-0123.nsys-rep [1/8] [========================98% ] nsys-report-0123.nsys-rep [1/8] [========================100%] nsys-report-0123.nsys-rep [1/8] [========================100%] nsys-report-0123.nsys-rep [2/8] [0% ] nsys-report-2c90.sqlite [2/8] [1% ] nsys-report-2c90.sqlite [2/8] [2% ] nsys-report-2c90.sqlite [2/8] [3% ] nsys-report-2c90.sqlite [2/8] [4% ] nsys-report-2c90.sqlite [2/8] [5% ] nsys-report-2c90.sqlite [2/8] [6% ] nsys-report-2c90.sqlite [2/8] [7% ] nsys-report-2c90.sqlite [2/8] [8% ] nsys-report-2c90.sqlite [2/8] [9% ] nsys-report-2c90.sqlite [2/8] [10% ] nsys-report-2c90.sqlite [2/8] [11% ] nsys-report-2c90.sqlite [2/8] [12% ] nsys-report-2c90.sqlite [2/8] [13% ] nsys-report-2c90.sqlite [2/8] [14% ] nsys-report-2c90.sqlite [2/8] [=15% ] nsys-report-2c90.sqlite [2/8] [=16% ] nsys-report-2c90.sqlite [2/8] [=17% ] nsys-report-2c90.sqlite [2/8] [==18% ] nsys-report-2c90.sqlite [2/8] [==19% ] nsys-report-2c90.sqlite [2/8] [==20% ] nsys-report-2c90.sqlite [2/8] [==21% ] nsys-report-2c90.sqlite [2/8] [===22% ] nsys-report-2c90.sqlite [2/8] [===23% ] nsys-report-2c90.sqlite [2/8] [===24% ] nsys-report-2c90.sqlite [2/8] [====25% ] nsys-report-2c90.sqlite [2/8] [====26% ] nsys-report-2c90.sqlite [2/8] [====27% ] nsys-report-2c90.sqlite [2/8] [====28% ] nsys-report-2c90.sqlite [2/8] [=====29% ] nsys-report-2c90.sqlite [2/8] [=====30% ] nsys-report-2c90.sqlite [2/8] [=====31% ] nsys-report-2c90.sqlite [2/8] [=====32% ] nsys-report-2c90.sqlite [2/8] [======33% ] nsys-report-2c90.sqlite [2/8] [======34% ] nsys-report-2c90.sqlite [2/8] [======35% ] nsys-report-2c90.sqlite [2/8] [=======36% ] nsys-report-2c90.sqlite [2/8] [=======37% ] nsys-report-2c90.sqlite [2/8] [=======38% ] nsys-report-2c90.sqlite [2/8] [=======39% ] nsys-report-2c90.sqlite [2/8] [========40% ] nsys-report-2c90.sqlite [2/8] [========41% ] nsys-report-2c90.sqlite [2/8] [========42% ] nsys-report-2c90.sqlite [2/8] [=========43% ] nsys-report-2c90.sqlite [2/8] [=========44% ] nsys-report-2c90.sqlite [2/8] [=========45% ] nsys-report-2c90.sqlite [2/8] [=========46% ] nsys-report-2c90.sqlite [2/8] [==========47% ] nsys-report-2c90.sqlite [2/8] [==========48% ] nsys-report-2c90.sqlite [2/8] [==========49% ] nsys-report-2c90.sqlite [2/8] [===========50% ] nsys-report-2c90.sqlite [2/8] [===========51% ] nsys-report-2c90.sqlite [2/8] [===========52% ] nsys-report-2c90.sqlite [2/8] [===========53% ] nsys-report-2c90.sqlite [2/8] [============54% ] nsys-report-2c90.sqlite [2/8] [============55% ] nsys-report-2c90.sqlite [2/8] [============56% ] nsys-report-2c90.sqlite [2/8] [============57% ] nsys-report-2c90.sqlite [2/8] [=============58% ] nsys-report-2c90.sqlite [2/8] [=============59% ] nsys-report-2c90.sqlite [2/8] [=============60% ] nsys-report-2c90.sqlite [2/8] [==============61% ] nsys-report-2c90.sqlite [2/8] [==============62% ] nsys-report-2c90.sqlite [2/8] [==============63% ] nsys-report-2c90.sqlite [2/8] [==============64% ] nsys-report-2c90.sqlite [2/8] [===============65% ] nsys-report-2c90.sqlite [2/8] [===============66% ] nsys-report-2c90.sqlite [2/8] [===============67% ] nsys-report-2c90.sqlite [2/8] [================68% ] nsys-report-2c90.sqlite [2/8] [================69% ] nsys-report-2c90.sqlite [2/8] [================70% ] nsys-report-2c90.sqlite [2/8] [================71% ] nsys-report-2c90.sqlite [2/8] [=================72% ] nsys-report-2c90.sqlite [2/8] [=================73% ] nsys-report-2c90.sqlite [2/8] [=================74% ] nsys-report-2c90.sqlite [2/8] [==================75% ] nsys-report-2c90.sqlite [2/8] [==================76% ] nsys-report-2c90.sqlite [2/8] [==================77% ] nsys-report-2c90.sqlite [2/8] [==================78% ] nsys-report-2c90.sqlite [2/8] [===================79% ] nsys-report-2c90.sqlite [2/8] [===================80% ] nsys-report-2c90.sqlite [2/8] [===================81% ] nsys-report-2c90.sqlite [2/8] [===================82% ] nsys-report-2c90.sqlite [2/8] [====================83% ] nsys-report-2c90.sqlite [2/8] [====================84% ] nsys-report-2c90.sqlite [2/8] [====================85% ] nsys-report-2c90.sqlite [2/8] [=====================86% ] nsys-report-2c90.sqlite [2/8] [=====================87% ] nsys-report-2c90.sqlite [2/8] [=====================88% ] nsys-report-2c90.sqlite [2/8] [=====================89% ] nsys-report-2c90.sqlite [2/8] [======================90% ] nsys-report-2c90.sqlite [2/8] [======================91% ] nsys-report-2c90.sqlite [2/8] [======================92% ] nsys-report-2c90.sqlite [2/8] [=======================93% ] nsys-report-2c90.sqlite [2/8] [=======================94% ] nsys-report-2c90.sqlite [2/8] [=======================95% ] nsys-report-2c90.sqlite [2/8] [=======================96% ] nsys-report-2c90.sqlite [2/8] [========================97% ] nsys-report-2c90.sqlite [2/8] [========================98% ] nsys-report-2c90.sqlite [2/8] [========================99% ] nsys-report-2c90.sqlite [2/8] [========================100%] nsys-report-2c90.sqlite [2/8] [========================100%] nsys-report-2c90.sqlite [3/8] Executing 'nvtx_sum' stats report [4/8] Executing 'osrt_sum' stats report Time (%) Total Time (ns) Num Calls Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name -------- --------------- --------- ------------- ------------- ----------- ----------- ------------ ---------------------- 55.2 5,230,820,475 63 83,028,896.4 100,119,708.0 1,280 548,595,156 73,482,897.6 poll 42.2 4,000,741,404 8 500,092,675.5 500,090,102.0 500,067,701 500,124,383 15,851.7 pthread_cond_timedwait 1.7 158,295,782 5,642 28,056.7 740.0 290 146,028,133 1,944,088.5 read 0.7 62,781,864 1,956 32,097.1 7,815.0 200 12,586,893 392,335.3 ioctl 0.1 9,348,135 3,183 2,936.9 2,700.0 1,140 36,151 1,256.3 open64 0.1 5,061,988 1 5,061,988.0 5,061,988.0 5,061,988 5,061,988 0.0 nanosleep 0.0 3,496,378 131,629 26.6 20.0 20 4,590 38.7 pthread_cond_signal 0.0 3,011,948 138 21,825.7 5,005.0 2,410 1,589,005 135,795.1 mmap64 0.0 2,619,399 71 36,892.9 930.0 580 672,060 111,420.4 pread64 0.0 809,893 13 62,299.5 59,751.0 54,861 80,871 8,354.2 sleep 0.0 515,469 583 884.2 50.0 20 60,211 5,485.3 fgets 0.0 481,325 28 17,190.2 6,820.0 1,760 114,061 23,885.3 mmap 0.0 377,505 10 37,750.5 35,975.5 15,230 78,841 20,343.3 sem_timedwait 0.0 348,995 8 43,624.4 31,290.0 25,720 72,221 20,452.4 pthread_create 0.0 221,624 29 7,642.2 2,700.0 490 52,390 12,795.7 write 0.0 204,595 44 4,649.9 2,830.5 970 21,260 4,496.3 fopen 0.0 174,132 10 17,413.2 4,120.0 1,940 79,041 29,324.5 munmap 0.0 130,762 1 130,762.0 130,762.0 130,762 130,762 0.0 pthread_cond_wait 0.0 101,411 1 101,411.0 101,411.0 101,411 101,411 0.0 waitpid 0.0 65,891 41 1,607.1 1,210.0 590 8,681 1,385.2 fclose 0.0 63,852 15 4,256.8 3,490.0 1,820 15,830 3,439.2 open 0.0 55,884 1,622 34.5 30.0 20 4,880 142.1 pthread_cond_broadcast 0.0 38,160 2 19,080.0 19,080.0 8,680 29,480 14,707.8 connect 0.0 33,752 6 5,625.3 5,155.5 2,960 10,060 2,994.9 pipe2 0.0 28,160 4 7,040.0 7,230.0 3,130 10,570 4,082.4 socket 0.0 27,231 133 204.7 210.0 20 1,210 133.4 sigaction 0.0 26,250 6 4,375.0 4,350.0 2,040 8,610 2,359.6 fopen64 0.0 22,074 68 324.6 300.0 171 1,090 148.3 fcntl 0.0 20,536 256 80.2 100.0 20 331 60.8 pthread_mutex_trylock 0.0 13,683 543 25.2 20.0 20 120 6.5 flockfile 0.0 13,500 3 4,500.0 4,280.0 1,530 7,690 3,085.9 fread 0.0 7,440 2 3,720.0 3,720.0 1,510 5,930 3,125.4 bind 0.0 3,279 30 109.3 30.0 20 880 189.0 fflush 0.0 2,620 10 262.0 255.0 220 350 37.9 dup 0.0 2,579 2 1,289.5 1,289.5 949 1,630 481.5 fwrite 0.0 1,180 2 590.0 590.0 380 800 297.0 dup2 0.0 890 1 890.0 890.0 890 890 0.0 getc 0.0 650 1 650.0 650.0 650 650 0.0 listen [5/8] Executing 'cuda_api_sum' stats report Time (%) Total Time (ns) Num Calls Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name -------- --------------- --------- ----------- ----------- --------- ---------- ----------- --------------------------------------------- 54.9 214,002,149 1,676 127,686.2 26,805.5 2,240 1,491,052 249,665.4 cudaMemcpyAsync 21.6 84,145,482 1,676 50,206.1 10,940.0 580 273,074 70,961.6 cudaStreamSynchronize 18.6 72,377,733 644 112,387.8 6,425.0 3,550 16,272,670 956,381.0 cudaLaunchKernel 2.0 7,716,799 2 3,858,399.5 3,858,399.5 1,150,218 6,566,581 3,829,947.0 cudaFree 1.6 6,338,147 9 704,238.6 1,370.0 290 6,327,717 2,108,804.6 cudaStreamIsCapturing_v10000 0.6 2,334,497 49 47,642.8 47,881.0 38,230 52,011 3,374.7 cuCtxSynchronize 0.2 893,243 9 99,249.2 102,492.0 4,411 158,362 54,393.9 cudaMalloc 0.2 657,800 49 13,424.5 13,550.0 8,540 27,110 2,774.6 cuLaunchKernel 0.1 318,881 62 5,143.2 3,580.0 2,800 16,960 2,706.9 cudaMemsetAsync 0.1 303,263 1,532 198.0 180.0 50 4,640 173.3 cuGetProcAddress_v2 0.1 218,475 2 109,237.5 109,237.5 73,382 145,093 50,707.3 cuModuleLoadData 0.0 161,342 1 161,342.0 161,342.0 161,342 161,342 0.0 cudaGetDeviceProperties_v2_v12000 0.0 75,655 26 2,909.8 2,545.0 349 7,140 1,384.1 cudaOccupancyMaxActiveBlocksPerMultiprocessor 0.0 18,840 18 1,046.7 265.0 140 10,740 2,478.0 cudaEventCreateWithFlags 0.0 4,670 1 4,670.0 4,670.0 4,670 4,670 0.0 cuMemFree_v2 0.0 3,880 4 970.0 970.0 580 1,360 369.7 cuInit 0.0 1,240 1 1,240.0 1,240.0 1,240 1,240 0.0 cuCtxSetCurrent 0.0 1,200 2 600.0 600.0 250 950 495.0 cudaGetDriverEntryPoint_v11030 0.0 560 4 140.0 115.0 80 250 76.2 cuModuleGetLoadingMode [6/8] Executing 'cuda_gpu_kern_sum' stats report Time (%) Total Time (ns) Instances Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name -------- --------------- --------- -------- -------- -------- -------- ----------- ---------------------------------------------------------------------------------------------------- 48.6 2,147,879 49 43,834.3 43,744.0 43,680 45,728 299.6 cutlass_tensorop_f16_s16816gemm_f16_256x128_32x3_tt_align8 7.4 326,528 48 6,802.7 6,752.0 6,560 7,136 178.6 void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl<at::native::… 6.8 299,042 12 24,920.2 24,896.5 24,800 25,248 115.0 ampere_fp16_s16816gemm_fp16_128x64_ldg8_f2f_stages_64x4_nn 4.6 204,162 72 2,835.6 2,880.0 2,336 3,424 376.7 void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl<at::native::… 4.0 175,617 48 3,658.7 3,744.0 3,392 3,904 226.8 void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<c10::Half, at::native::MeanOp… 3.3 143,967 12 11,997.3 11,952.0 11,872 12,543 175.8 void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl<at::native::… 2.9 126,912 24 5,288.0 5,328.0 4,832 5,696 281.0 void cutlass::Kernel<cutlass_80_wmma_tensorop_f16_s161616gemm_f16_32x32_64x1_nn_align8>(T1::Params) 2.3 102,977 48 2,145.4 2,176.0 1,984 2,368 134.6 void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl<at::native::… 2.3 100,766 36 2,799.1 2,240.0 2,207 4,000 821.3 void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl<at::native::… 2.2 96,065 12 8,005.4 8,032.0 7,713 8,161 109.7 void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl<at::native::… 2.1 94,881 12 7,906.8 7,904.0 7,809 7,999 53.2 ampere_fp16_s16816gemm_fp16_64x64_ldg8_f2f_stages_64x5_nn 1.9 84,862 12 7,071.8 7,103.0 6,752 7,168 110.8 void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl<at::native::… 1.5 65,501 36 1,819.5 1,472.0 1,440 2,560 493.3 void at::native::vectorized_elementwise_kernel<(int)4, at::native::CUDAFunctor_add<c10::Half>, at::… 1.5 65,190 36 1,810.8 1,473.0 1,409 2,528 495.9 void at::native::vectorized_elementwise_kernel<(int)4, at::native::BinaryFunctor<c10::Half, c10::Ha… 1.5 64,862 27 2,402.3 2,432.0 1,824 2,560 131.9 void at::native::elementwise_kernel<(int)128, (int)2, void at::native::gpu_kernel_impl<at::native::… 1.5 64,159 12 5,346.6 5,280.0 5,216 6,016 218.5 void <unnamed>::softmax_warp_forward<float, float, float, (int)8, (bool)0, (bool)0>(T2 *, const T1 … 1.4 61,535 49 1,255.8 1,088.0 800 1,888 340.6 void at::native::vectorized_elementwise_kernel<(int)4, at::native::FillFunctor<c10::Half>, at::deta… 0.7 31,456 24 1,310.7 1,312.0 1,216 1,344 22.0 void at::native::unrolled_elementwise_kernel<at::native::CUDAFunctor_add<c10::Half>, at::detail::Ar… 0.7 30,944 12 2,578.7 2,528.0 2,528 3,040 146.0 void at::native::vectorized_elementwise_kernel<(int)4, at::native::tanh_kernel_cuda(at::TensorItera… 0.6 27,519 27 1,019.2 992.0 928 1,824 161.4 void at::native::vectorized_elementwise_kernel<(int)4, at::native::CUDAFunctor_add<float>, at::deta… 0.6 27,229 24 1,134.5 1,151.0 1,056 1,153 23.0 void at::native::vectorized_elementwise_kernel<(int)4, at::native::reciprocal_kernel_cuda(at::Tenso… 0.6 26,368 24 1,098.7 1,088.0 1,056 1,120 18.1 void at::native::vectorized_elementwise_kernel<(int)4, at::native::sqrt_kernel_cuda(at::TensorItera… 0.5 22,369 24 932.0 928.0 896 960 14.3 void at::native::vectorized_elementwise_kernel<(int)4, at::native::AUnaryFunctor<c10::Half, c10::Ha… 0.2 9,408 5 1,881.6 1,632.0 1,344 2,464 527.5 void at::native::elementwise_kernel<(int)128, (int)2, void at::native::gpu_kernel_impl<at::native::… 0.2 7,808 2 3,904.0 3,904.0 3,744 4,064 226.3 void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<float, at::native::MeanOps<fl… 0.1 2,753 1 2,753.0 2,753.0 2,753 2,753 0.0 void at::native::unrolled_elementwise_kernel<at::native::CUDAFunctor_add<float>, at::detail::Array<… 0.0 2,144 1 2,144.0 2,144.0 2,144 2,144 0.0 void cutlass::Kernel<cutlass_80_wmma_tensorop_f16_s161616gemm_f16_32x32_32x1_nn_align2>(T1::Params) 0.0 1,856 1 1,856.0 1,856.0 1,856 1,856 0.0 void at::native::vectorized_elementwise_kernel<(int)4, at::native::BinaryFunctor<float, float, floa… 0.0 1,024 1 1,024.0 1,024.0 1,024 1,024 0.0 void at::native::vectorized_elementwise_kernel<(int)4, at::native::reciprocal_kernel_cuda(at::Tenso… 0.0 991 1 991.0 991.0 991 991 0.0 void at::native::vectorized_elementwise_kernel<(int)4, at::native::sqrt_kernel_cuda(at::TensorItera… 0.0 928 1 928.0 928.0 928 928 0.0 void at::native::vectorized_elementwise_kernel<(int)4, at::native::AUnaryFunctor<float, float, floa… [7/8] Executing 'cuda_gpu_mem_time_sum' stats report Time (%) Total Time (ns) Count Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Operation -------- --------------- ----- --------- -------- -------- -------- ----------- ---------------------------- 59.5 96,180,030 1,092 88,077.0 61,152.0 287 754,659 134,057.1 [CUDA memcpy Host-to-Device] 40.5 65,543,200 584 112,231.5 59,296.0 960 809,219 154,864.6 [CUDA memcpy Device-to-Host] 0.0 29,726 62 479.5 320.0 288 928 232.2 [CUDA memset] [8/8] Executing 'cuda_gpu_mem_size_sum' stats report Total (MB) Count Avg (MB) Med (MB) Min (MB) Max (MB) StdDev (MB) Operation ---------- ----- -------- -------- -------- -------- ----------- ---------------------------- 626.375 1,092 0.574 0.393 0.000 4.719 0.881 [CUDA memcpy Host-to-Device] 393.055 584 0.673 0.393 0.000 3.146 0.785 [CUDA memcpy Device-to-Host] 0.001 62 0.000 0.000 0.000 0.000 0.000 [CUDA memset] Generated: /tmp/nsys-report-0123.nsys-rep /tmp/nsys-report-2c90.sqlite
Editor is loading...
Leave a Comment