Untitled
unknown
c_cpp
2 months ago
81 kB
1
Indexable
Model Input Name: unique_ids_raw_output___9:0, Shape: [0] Model Input Name: segment_ids:0, Shape: [0, 256] Model Input Name: input_mask:0, Shape: [0, 256] Model Input Name: input_ids:0, Shape: [0, 256] Starting model execution... Inputs Details: Input Name: input_ids:0 Shape: (1, 256) Data (first 10 values): [25837 24066 17917 10287 4815 12979 20169 8864 10101 23255]... -------------------------------------------------- Input Name: segment_ids:0 Shape: (1, 256) Data (first 10 values): [0 0 0 0 0 1 1 1 1 0]... -------------------------------------------------- Input Name: input_mask:0 Shape: (1, 256) Data (first 10 values): [0 0 1 0 1 1 0 0 0 1]... -------------------------------------------------- Input Name: unique_ids_raw_output___9:0 Shape: (0,) Data (first 10 values): []... -------------------------------------------------- No Add node related to MatMul output: bert/embeddings/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_0/attention/self/value/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_0/attention/self/query/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_0/attention/self/key/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_0/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_0/attention/self/MatMul_1. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_0/attention/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_0/intermediate/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_0/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_1/attention/self/value/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_1/attention/self/query/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_1/attention/self/key/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_1/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_1/attention/self/MatMul_1. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_1/attention/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_1/intermediate/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_1/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_2/attention/self/value/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_2/attention/self/query/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_2/attention/self/key/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_2/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_2/attention/self/MatMul_1. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_2/attention/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_2/intermediate/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_2/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_3/attention/self/value/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_3/attention/self/query/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_3/attention/self/key/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_3/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_3/attention/self/MatMul_1. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_3/attention/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_3/intermediate/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_3/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_4/attention/self/value/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_4/attention/self/query/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_4/attention/self/key/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_4/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_4/attention/self/MatMul_1. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_4/attention/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_4/intermediate/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_4/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_5/attention/self/value/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_5/attention/self/query/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_5/attention/self/key/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_5/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_5/attention/self/MatMul_1. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_5/attention/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_5/intermediate/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_5/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_6/attention/self/value/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_6/attention/self/query/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_6/attention/self/key/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_6/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_6/attention/self/MatMul_1. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_6/attention/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_6/intermediate/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_6/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_7/attention/self/value/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_7/attention/self/query/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_7/attention/self/key/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_7/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_7/attention/self/MatMul_1. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_7/attention/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_7/intermediate/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_7/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_8/attention/self/value/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_8/attention/self/query/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_8/attention/self/key/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_8/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_8/attention/self/MatMul_1. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_8/attention/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_8/intermediate/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_8/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_9/attention/self/value/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_9/attention/self/query/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_9/attention/self/key/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_9/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_9/attention/self/MatMul_1. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_9/attention/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_9/intermediate/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_9/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_10/attention/self/value/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_10/attention/self/query/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_10/attention/self/key/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_10/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_10/attention/self/MatMul_1. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_10/attention/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_10/intermediate/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_10/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_11/attention/self/value/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_11/attention/self/query/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_11/attention/self/key/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_11/attention/self/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_11/attention/self/MatMul_1. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_11/attention/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_11/intermediate/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: bert/encoder/layer_11/output/dense/MatMul. Executing regular MatMul. No Add node related to MatMul output: MatMul. Executing regular MatMul. Node Execution Times: Node: unique_ids_graph_outputs_Identity__10, Execution Time: 0.000005 seconds Node: bert/encoder/Shape, Execution Time: 0.000004 seconds Node: bert/encoder/Shape__12, Execution Time: 0.000009 seconds Node: bert/encoder/strided_slice, Execution Time: 0.000062 seconds Node: bert/encoder/strided_slice__16, Execution Time: 0.000005 seconds Node: bert/encoder/strided_slice__17, Execution Time: 0.000005 seconds Node: bert/encoder/ones/packed_Unsqueeze__18, Execution Time: 0.000010 seconds Node: bert/encoder/ones/packed_Concat__21, Execution Time: 0.000010 seconds Node: bert/encoder/ones__22, Execution Time: 0.000004 seconds Node: bert/encoder/ones, Execution Time: 0.000010 seconds Node: bert/encoder/Reshape, Execution Time: 0.000006 seconds Node: bert/encoder/Cast, Execution Time: 0.000004 seconds Node: bert/encoder/mul, Execution Time: 0.028551 seconds Node: bert/encoder/layer_9/attention/self/ExpandDims, Execution Time: 0.000035 seconds Node: bert/encoder/layer_9/attention/self/sub, Execution Time: 0.006637 seconds Node: bert/encoder/layer_9/attention/self/mul_1, Execution Time: 0.000351 seconds Node: bert/embeddings/Reshape_2, Execution Time: 0.000028 seconds Node: bert/embeddings/Reshape, Execution Time: 0.000006 seconds Node: bert/embeddings/GatherV2, Execution Time: 0.000317 seconds Node: bert/embeddings/Reshape_1, Execution Time: 0.000007 seconds Node: bert/embeddings/one_hot, Execution Time: 0.000059 seconds Node: bert/embeddings/MatMul, Execution Time: 0.059774 seconds Node: bert/embeddings/Reshape_3, Execution Time: 0.000044 seconds Node: bert/embeddings/add, Execution Time: 0.002200 seconds Node: bert/embeddings/add_1, Execution Time: 0.001008 seconds Node: bert/embeddings/LayerNorm/moments/mean, Execution Time: 0.005287 seconds Node: bert/embeddings/LayerNorm/moments/SquaredDifference, Execution Time: 0.000715 seconds Node: bert/embeddings/LayerNorm/moments/SquaredDifference__72, Execution Time: 0.000848 seconds Node: bert/embeddings/LayerNorm/moments/variance, Execution Time: 0.000269 seconds Node: bert/embeddings/LayerNorm/batchnorm/add, Execution Time: 0.000084 seconds Node: bert/embeddings/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.009919 seconds Node: bert/embeddings/LayerNorm/batchnorm/Rsqrt__74, Execution Time: 0.005503 seconds Node: bert/embeddings/LayerNorm/batchnorm/mul, Execution Time: 0.000118 seconds Node: bert/embeddings/LayerNorm/batchnorm/mul_2, Execution Time: 0.000081 seconds Node: bert/embeddings/LayerNorm/batchnorm/sub, Execution Time: 0.000090 seconds Node: bert/embeddings/LayerNorm/batchnorm/mul_1, Execution Time: 0.000676 seconds Node: bert/embeddings/LayerNorm/batchnorm/add_1, Execution Time: 0.000734 seconds Node: bert/encoder/Reshape_1, Execution Time: 0.000036 seconds Node: bert/encoder/layer_0/attention/self/value/MatMul, Execution Time: 0.001673 seconds Node: bert/encoder/layer_0/attention/self/value/BiasAdd, Execution Time: 0.000724 seconds Node: bert/encoder/layer_0/attention/self/Reshape_2, Execution Time: 0.000018 seconds Node: bert/encoder/layer_0/attention/self/transpose_2, Execution Time: 0.000760 seconds Node: bert/encoder/layer_0/attention/self/query/MatMul, Execution Time: 0.001025 seconds Node: bert/encoder/layer_0/attention/self/query/BiasAdd, Execution Time: 0.000627 seconds Node: bert/encoder/layer_0/attention/self/Reshape, Execution Time: 0.000014 seconds Node: bert/encoder/layer_0/attention/self/transpose, Execution Time: 0.000697 seconds Node: bert/encoder/layer_0/attention/self/key/MatMul, Execution Time: 0.000985 seconds Node: bert/encoder/layer_0/attention/self/key/BiasAdd, Execution Time: 0.000647 seconds Node: bert/encoder/layer_0/attention/self/Reshape_1, Execution Time: 0.000014 seconds Node: bert/encoder/layer_0/attention/self/MatMul__306, Execution Time: 0.000694 seconds Node: bert/encoder/layer_0/attention/self/MatMul, Execution Time: 0.005266 seconds Node: bert/encoder/layer_0/attention/self/Mul, Execution Time: 0.001311 seconds Node: bert/encoder/layer_0/attention/self/add, Execution Time: 0.001939 seconds Node: bert/encoder/layer_0/attention/self/Softmax, Execution Time: 0.009698 seconds Node: bert/encoder/layer_0/attention/self/MatMul_1, Execution Time: 0.001546 seconds Node: bert/encoder/layer_0/attention/self/transpose_3, Execution Time: 0.000404 seconds Node: bert/encoder/layer_0/attention/self/Reshape_3, Execution Time: 0.000089 seconds Node: bert/encoder/layer_0/attention/output/dense/MatMul, Execution Time: 0.000978 seconds Node: bert/encoder/layer_0/attention/output/dense/BiasAdd, Execution Time: 0.000740 seconds Node: bert/encoder/layer_0/attention/output/add, Execution Time: 0.000922 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/moments/mean, Execution Time: 0.000290 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000698 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/moments/SquaredDifference__309, Execution Time: 0.000864 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/moments/variance, Execution Time: 0.000259 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000088 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000071 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/Rsqrt__311, Execution Time: 0.000094 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000079 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000075 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000076 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000643 seconds Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000675 seconds Node: bert/encoder/layer_0/intermediate/dense/MatMul, Execution Time: 0.002977 seconds Node: bert/encoder/layer_0/intermediate/dense/BiasAdd, Execution Time: 0.001722 seconds Node: bert/encoder/layer_0/intermediate/dense/Pow, Execution Time: 0.017356 seconds Node: bert/encoder/layer_0/intermediate/dense/mul, Execution Time: 0.002039 seconds Node: bert/encoder/layer_0/intermediate/dense/add, Execution Time: 0.002510 seconds Node: bert/encoder/layer_0/intermediate/dense/mul_1, Execution Time: 0.001973 seconds Node: bert/encoder/layer_0/intermediate/dense/Tanh, Execution Time: 0.004290 seconds Node: bert/encoder/layer_0/intermediate/dense/add_1, Execution Time: 0.001899 seconds Node: bert/encoder/layer_0/intermediate/dense/mul_2, Execution Time: 0.001950 seconds Node: bert/encoder/layer_0/intermediate/dense/mul_3, Execution Time: 0.002503 seconds Node: bert/encoder/layer_0/output/dense/MatMul, Execution Time: 0.002916 seconds Node: bert/encoder/layer_0/output/dense/BiasAdd, Execution Time: 0.000371 seconds Node: bert/encoder/layer_0/output/add, Execution Time: 0.000650 seconds Node: bert/encoder/layer_0/output/LayerNorm/moments/mean, Execution Time: 0.000284 seconds Node: bert/encoder/layer_0/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000421 seconds Node: bert/encoder/layer_0/output/LayerNorm/moments/SquaredDifference__313, Execution Time: 0.000587 seconds Node: bert/encoder/layer_0/output/LayerNorm/moments/variance, Execution Time: 0.000247 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/add, Execution Time: 0.000197 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000068 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/Rsqrt__315, Execution Time: 0.000093 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/mul, Execution Time: 0.000093 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000141 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/sub, Execution Time: 0.000108 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000391 seconds Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000437 seconds Node: bert/encoder/layer_1/attention/self/value/MatMul, Execution Time: 0.000922 seconds Node: bert/encoder/layer_1/attention/self/value/BiasAdd, Execution Time: 0.000615 seconds Node: bert/encoder/layer_1/attention/self/Reshape_2, Execution Time: 0.000031 seconds Node: bert/encoder/layer_1/attention/self/transpose_2, Execution Time: 0.000656 seconds Node: bert/encoder/layer_1/attention/self/query/MatMul, Execution Time: 0.000841 seconds Node: bert/encoder/layer_1/attention/self/query/BiasAdd, Execution Time: 0.000588 seconds Node: bert/encoder/layer_1/attention/self/Reshape, Execution Time: 0.000013 seconds Node: bert/encoder/layer_1/attention/self/transpose, Execution Time: 0.000686 seconds Node: bert/encoder/layer_1/attention/self/key/MatMul, Execution Time: 0.000851 seconds Node: bert/encoder/layer_1/attention/self/key/BiasAdd, Execution Time: 0.000602 seconds Node: bert/encoder/layer_1/attention/self/Reshape_1, Execution Time: 0.000011 seconds Node: bert/encoder/layer_1/attention/self/MatMul__320, Execution Time: 0.000679 seconds Node: bert/encoder/layer_1/attention/self/MatMul, Execution Time: 0.001524 seconds Node: bert/encoder/layer_1/attention/self/Mul, Execution Time: 0.001187 seconds Node: bert/encoder/layer_1/attention/self/add, Execution Time: 0.001781 seconds Node: bert/encoder/layer_1/attention/self/Softmax, Execution Time: 0.002000 seconds Node: bert/encoder/layer_1/attention/self/MatMul_1, Execution Time: 0.001091 seconds Node: bert/encoder/layer_1/attention/self/transpose_3, Execution Time: 0.000237 seconds Node: bert/encoder/layer_1/attention/self/Reshape_3, Execution Time: 0.000040 seconds Node: bert/encoder/layer_1/attention/output/dense/MatMul, Execution Time: 0.000849 seconds Node: bert/encoder/layer_1/attention/output/dense/BiasAdd, Execution Time: 0.000601 seconds Node: bert/encoder/layer_1/attention/output/add, Execution Time: 0.000849 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/moments/mean, Execution Time: 0.000266 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000650 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/moments/SquaredDifference__323, Execution Time: 0.000845 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/moments/variance, Execution Time: 0.000244 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000082 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000057 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/Rsqrt__325, Execution Time: 0.000082 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000079 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000074 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000074 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000645 seconds Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000679 seconds Node: bert/encoder/layer_1/intermediate/dense/MatMul, Execution Time: 0.002089 seconds Node: bert/encoder/layer_1/intermediate/dense/BiasAdd, Execution Time: 0.001832 seconds Node: bert/encoder/layer_1/intermediate/dense/Pow, Execution Time: 0.002025 seconds Node: bert/encoder/layer_1/intermediate/dense/mul, Execution Time: 0.002000 seconds Node: bert/encoder/layer_1/intermediate/dense/add, Execution Time: 0.002538 seconds Node: bert/encoder/layer_1/intermediate/dense/mul_1, Execution Time: 0.001912 seconds Node: bert/encoder/layer_1/intermediate/dense/Tanh, Execution Time: 0.001834 seconds Node: bert/encoder/layer_1/intermediate/dense/add_1, Execution Time: 0.001891 seconds Node: bert/encoder/layer_1/intermediate/dense/mul_2, Execution Time: 0.001838 seconds Node: bert/encoder/layer_1/intermediate/dense/mul_3, Execution Time: 0.002411 seconds Node: bert/encoder/layer_1/output/dense/MatMul, Execution Time: 0.002572 seconds Node: bert/encoder/layer_1/output/dense/BiasAdd, Execution Time: 0.000360 seconds Node: bert/encoder/layer_1/output/add, Execution Time: 0.000635 seconds Node: bert/encoder/layer_1/output/LayerNorm/moments/mean, Execution Time: 0.000263 seconds Node: bert/encoder/layer_1/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000395 seconds Node: bert/encoder/layer_1/output/LayerNorm/moments/SquaredDifference__327, Execution Time: 0.000609 seconds Node: bert/encoder/layer_1/output/LayerNorm/moments/variance, Execution Time: 0.000252 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/add, Execution Time: 0.000079 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000058 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/Rsqrt__329, Execution Time: 0.000086 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/mul, Execution Time: 0.000079 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000102 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/sub, Execution Time: 0.000099 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000505 seconds Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000581 seconds Node: bert/encoder/layer_2/attention/self/value/MatMul, Execution Time: 0.001113 seconds Node: bert/encoder/layer_2/attention/self/value/BiasAdd, Execution Time: 0.000770 seconds Node: bert/encoder/layer_2/attention/self/Reshape_2, Execution Time: 0.000047 seconds Node: bert/encoder/layer_2/attention/self/transpose_2, Execution Time: 0.000836 seconds Node: bert/encoder/layer_2/attention/self/query/MatMul, Execution Time: 0.001082 seconds Node: bert/encoder/layer_2/attention/self/query/BiasAdd, Execution Time: 0.000746 seconds Node: bert/encoder/layer_2/attention/self/Reshape, Execution Time: 0.000013 seconds Node: bert/encoder/layer_2/attention/self/transpose, Execution Time: 0.000850 seconds Node: bert/encoder/layer_2/attention/self/key/MatMul, Execution Time: 0.001104 seconds Node: bert/encoder/layer_2/attention/self/key/BiasAdd, Execution Time: 0.000748 seconds Node: bert/encoder/layer_2/attention/self/Reshape_1, Execution Time: 0.000011 seconds Node: bert/encoder/layer_2/attention/self/MatMul__334, Execution Time: 0.000670 seconds Node: bert/encoder/layer_2/attention/self/MatMul, Execution Time: 0.001525 seconds Node: bert/encoder/layer_2/attention/self/Mul, Execution Time: 0.001232 seconds Node: bert/encoder/layer_2/attention/self/add, Execution Time: 0.001778 seconds Node: bert/encoder/layer_2/attention/self/Softmax, Execution Time: 0.001995 seconds Node: bert/encoder/layer_2/attention/self/MatMul_1, Execution Time: 0.001145 seconds Node: bert/encoder/layer_2/attention/self/transpose_3, Execution Time: 0.000355 seconds Node: bert/encoder/layer_2/attention/self/Reshape_3, Execution Time: 0.000037 seconds Node: bert/encoder/layer_2/attention/output/dense/MatMul, Execution Time: 0.000839 seconds Node: bert/encoder/layer_2/attention/output/dense/BiasAdd, Execution Time: 0.000597 seconds Node: bert/encoder/layer_2/attention/output/add, Execution Time: 0.000865 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/moments/mean, Execution Time: 0.000256 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000659 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/moments/SquaredDifference__337, Execution Time: 0.000824 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/moments/variance, Execution Time: 0.000239 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000078 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000056 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/Rsqrt__339, Execution Time: 0.000093 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000080 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000073 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000076 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000648 seconds Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000679 seconds Node: bert/encoder/layer_2/intermediate/dense/MatMul, Execution Time: 0.002191 seconds Node: bert/encoder/layer_2/intermediate/dense/BiasAdd, Execution Time: 0.001873 seconds Node: bert/encoder/layer_2/intermediate/dense/Pow, Execution Time: 0.001935 seconds Node: bert/encoder/layer_2/intermediate/dense/mul, Execution Time: 0.001916 seconds Node: bert/encoder/layer_2/intermediate/dense/add, Execution Time: 0.002496 seconds Node: bert/encoder/layer_2/intermediate/dense/mul_1, Execution Time: 0.001880 seconds Node: bert/encoder/layer_2/intermediate/dense/Tanh, Execution Time: 0.001877 seconds Node: bert/encoder/layer_2/intermediate/dense/add_1, Execution Time: 0.001864 seconds Node: bert/encoder/layer_2/intermediate/dense/mul_2, Execution Time: 0.001848 seconds Node: bert/encoder/layer_2/intermediate/dense/mul_3, Execution Time: 0.002449 seconds Node: bert/encoder/layer_2/output/dense/MatMul, Execution Time: 0.002691 seconds Node: bert/encoder/layer_2/output/dense/BiasAdd, Execution Time: 0.000410 seconds Node: bert/encoder/layer_2/output/add, Execution Time: 0.000596 seconds Node: bert/encoder/layer_2/output/LayerNorm/moments/mean, Execution Time: 0.000255 seconds Node: bert/encoder/layer_2/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000392 seconds Node: bert/encoder/layer_2/output/LayerNorm/moments/SquaredDifference__341, Execution Time: 0.000555 seconds Node: bert/encoder/layer_2/output/LayerNorm/moments/variance, Execution Time: 0.000290 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/add, Execution Time: 0.000079 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/Rsqrt__343, Execution Time: 0.000080 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/mul, Execution Time: 0.000076 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000077 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/sub, Execution Time: 0.000083 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000401 seconds Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000427 seconds Node: bert/encoder/layer_3/attention/self/value/MatMul, Execution Time: 0.000859 seconds Node: bert/encoder/layer_3/attention/self/value/BiasAdd, Execution Time: 0.000601 seconds Node: bert/encoder/layer_3/attention/self/Reshape_2, Execution Time: 0.000030 seconds Node: bert/encoder/layer_3/attention/self/transpose_2, Execution Time: 0.000668 seconds Node: bert/encoder/layer_3/attention/self/query/MatMul, Execution Time: 0.000854 seconds Node: bert/encoder/layer_3/attention/self/query/BiasAdd, Execution Time: 0.000878 seconds Node: bert/encoder/layer_3/attention/self/Reshape, Execution Time: 0.000024 seconds Node: bert/encoder/layer_3/attention/self/transpose, Execution Time: 0.000851 seconds Node: bert/encoder/layer_3/attention/self/key/MatMul, Execution Time: 0.000908 seconds Node: bert/encoder/layer_3/attention/self/key/BiasAdd, Execution Time: 0.000679 seconds Node: bert/encoder/layer_3/attention/self/Reshape_1, Execution Time: 0.000014 seconds Node: bert/encoder/layer_3/attention/self/MatMul__348, Execution Time: 0.000738 seconds Node: bert/encoder/layer_3/attention/self/MatMul, Execution Time: 0.001674 seconds Node: bert/encoder/layer_3/attention/self/Mul, Execution Time: 0.001269 seconds Node: bert/encoder/layer_3/attention/self/add, Execution Time: 0.002073 seconds Node: bert/encoder/layer_3/attention/self/Softmax, Execution Time: 0.002213 seconds Node: bert/encoder/layer_3/attention/self/MatMul_1, Execution Time: 0.001187 seconds Node: bert/encoder/layer_3/attention/self/transpose_3, Execution Time: 0.000427 seconds Node: bert/encoder/layer_3/attention/self/Reshape_3, Execution Time: 0.000047 seconds Node: bert/encoder/layer_3/attention/output/dense/MatMul, Execution Time: 0.000892 seconds Node: bert/encoder/layer_3/attention/output/dense/BiasAdd, Execution Time: 0.000687 seconds Node: bert/encoder/layer_3/attention/output/add, Execution Time: 0.000955 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/moments/mean, Execution Time: 0.000285 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000771 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/moments/SquaredDifference__351, Execution Time: 0.000914 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/moments/variance, Execution Time: 0.000256 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000089 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000065 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/Rsqrt__353, Execution Time: 0.000097 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000136 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000095 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000089 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000765 seconds Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000893 seconds Node: bert/encoder/layer_3/intermediate/dense/MatMul, Execution Time: 0.002253 seconds Node: bert/encoder/layer_3/intermediate/dense/BiasAdd, Execution Time: 0.001940 seconds Node: bert/encoder/layer_3/intermediate/dense/Pow, Execution Time: 0.002200 seconds Node: bert/encoder/layer_3/intermediate/dense/mul, Execution Time: 0.002178 seconds Node: bert/encoder/layer_3/intermediate/dense/add, Execution Time: 0.003325 seconds Node: bert/encoder/layer_3/intermediate/dense/mul_1, Execution Time: 0.002681 seconds Node: bert/encoder/layer_3/intermediate/dense/Tanh, Execution Time: 0.002738 seconds Node: bert/encoder/layer_3/intermediate/dense/add_1, Execution Time: 0.002450 seconds Node: bert/encoder/layer_3/intermediate/dense/mul_2, Execution Time: 0.001890 seconds Node: bert/encoder/layer_3/intermediate/dense/mul_3, Execution Time: 0.002413 seconds Node: bert/encoder/layer_3/output/dense/MatMul, Execution Time: 0.002623 seconds Node: bert/encoder/layer_3/output/dense/BiasAdd, Execution Time: 0.000356 seconds Node: bert/encoder/layer_3/output/add, Execution Time: 0.000618 seconds Node: bert/encoder/layer_3/output/LayerNorm/moments/mean, Execution Time: 0.000263 seconds Node: bert/encoder/layer_3/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000425 seconds Node: bert/encoder/layer_3/output/LayerNorm/moments/SquaredDifference__355, Execution Time: 0.000579 seconds Node: bert/encoder/layer_3/output/LayerNorm/moments/variance, Execution Time: 0.000248 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/add, Execution Time: 0.000081 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000058 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/Rsqrt__357, Execution Time: 0.000084 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/mul, Execution Time: 0.000079 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000075 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/sub, Execution Time: 0.000077 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000415 seconds Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000434 seconds Node: bert/encoder/layer_4/attention/self/value/MatMul, Execution Time: 0.000905 seconds Node: bert/encoder/layer_4/attention/self/value/BiasAdd, Execution Time: 0.000623 seconds Node: bert/encoder/layer_4/attention/self/Reshape_2, Execution Time: 0.000029 seconds Node: bert/encoder/layer_4/attention/self/transpose_2, Execution Time: 0.000681 seconds Node: bert/encoder/layer_4/attention/self/query/MatMul, Execution Time: 0.000915 seconds Node: bert/encoder/layer_4/attention/self/query/BiasAdd, Execution Time: 0.000590 seconds Node: bert/encoder/layer_4/attention/self/Reshape, Execution Time: 0.000020 seconds Node: bert/encoder/layer_4/attention/self/transpose, Execution Time: 0.000654 seconds Node: bert/encoder/layer_4/attention/self/key/MatMul, Execution Time: 0.000882 seconds Node: bert/encoder/layer_4/attention/self/key/BiasAdd, Execution Time: 0.000594 seconds Node: bert/encoder/layer_4/attention/self/Reshape_1, Execution Time: 0.000019 seconds Node: bert/encoder/layer_4/attention/self/MatMul__362, Execution Time: 0.000650 seconds Node: bert/encoder/layer_4/attention/self/MatMul, Execution Time: 0.001553 seconds Node: bert/encoder/layer_4/attention/self/Mul, Execution Time: 0.001226 seconds Node: bert/encoder/layer_4/attention/self/add, Execution Time: 0.001808 seconds Node: bert/encoder/layer_4/attention/self/Softmax, Execution Time: 0.002079 seconds Node: bert/encoder/layer_4/attention/self/MatMul_1, Execution Time: 0.001123 seconds Node: bert/encoder/layer_4/attention/self/transpose_3, Execution Time: 0.000372 seconds Node: bert/encoder/layer_4/attention/self/Reshape_3, Execution Time: 0.000042 seconds Node: bert/encoder/layer_4/attention/output/dense/MatMul, Execution Time: 0.000882 seconds Node: bert/encoder/layer_4/attention/output/dense/BiasAdd, Execution Time: 0.000640 seconds Node: bert/encoder/layer_4/attention/output/add, Execution Time: 0.000884 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/moments/mean, Execution Time: 0.000257 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000662 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/moments/SquaredDifference__365, Execution Time: 0.000813 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/moments/variance, Execution Time: 0.000248 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000080 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000059 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/Rsqrt__367, Execution Time: 0.000079 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000083 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000077 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000088 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000659 seconds Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000667 seconds Node: bert/encoder/layer_4/intermediate/dense/MatMul, Execution Time: 0.002138 seconds Node: bert/encoder/layer_4/intermediate/dense/BiasAdd, Execution Time: 0.001744 seconds Node: bert/encoder/layer_4/intermediate/dense/Pow, Execution Time: 0.001949 seconds Node: bert/encoder/layer_4/intermediate/dense/mul, Execution Time: 0.001999 seconds Node: bert/encoder/layer_4/intermediate/dense/add, Execution Time: 0.002471 seconds Node: bert/encoder/layer_4/intermediate/dense/mul_1, Execution Time: 0.001936 seconds Node: bert/encoder/layer_4/intermediate/dense/Tanh, Execution Time: 0.001891 seconds Node: bert/encoder/layer_4/intermediate/dense/add_1, Execution Time: 0.001945 seconds Node: bert/encoder/layer_4/intermediate/dense/mul_2, Execution Time: 0.001934 seconds Node: bert/encoder/layer_4/intermediate/dense/mul_3, Execution Time: 0.002528 seconds Node: bert/encoder/layer_4/output/dense/MatMul, Execution Time: 0.002568 seconds Node: bert/encoder/layer_4/output/dense/BiasAdd, Execution Time: 0.000357 seconds Node: bert/encoder/layer_4/output/add, Execution Time: 0.000647 seconds Node: bert/encoder/layer_4/output/LayerNorm/moments/mean, Execution Time: 0.000253 seconds Node: bert/encoder/layer_4/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000395 seconds Node: bert/encoder/layer_4/output/LayerNorm/moments/SquaredDifference__369, Execution Time: 0.000578 seconds Node: bert/encoder/layer_4/output/LayerNorm/moments/variance, Execution Time: 0.000245 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/add, Execution Time: 0.000077 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000057 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/Rsqrt__371, Execution Time: 0.000081 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/mul, Execution Time: 0.000076 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000073 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/sub, Execution Time: 0.000080 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000385 seconds Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000447 seconds Node: bert/encoder/layer_5/attention/self/value/MatMul, Execution Time: 0.000831 seconds Node: bert/encoder/layer_5/attention/self/value/BiasAdd, Execution Time: 0.000592 seconds Node: bert/encoder/layer_5/attention/self/Reshape_2, Execution Time: 0.000030 seconds Node: bert/encoder/layer_5/attention/self/transpose_2, Execution Time: 0.000645 seconds Node: bert/encoder/layer_5/attention/self/query/MatMul, Execution Time: 0.000823 seconds Node: bert/encoder/layer_5/attention/self/query/BiasAdd, Execution Time: 0.000595 seconds Node: bert/encoder/layer_5/attention/self/Reshape, Execution Time: 0.000012 seconds Node: bert/encoder/layer_5/attention/self/transpose, Execution Time: 0.000641 seconds Node: bert/encoder/layer_5/attention/self/key/MatMul, Execution Time: 0.000831 seconds Node: bert/encoder/layer_5/attention/self/key/BiasAdd, Execution Time: 0.000592 seconds Node: bert/encoder/layer_5/attention/self/Reshape_1, Execution Time: 0.000011 seconds Node: bert/encoder/layer_5/attention/self/MatMul__376, Execution Time: 0.000674 seconds Node: bert/encoder/layer_5/attention/self/MatMul, Execution Time: 0.001515 seconds Node: bert/encoder/layer_5/attention/self/Mul, Execution Time: 0.001233 seconds Node: bert/encoder/layer_5/attention/self/add, Execution Time: 0.001842 seconds Node: bert/encoder/layer_5/attention/self/Softmax, Execution Time: 0.002027 seconds Node: bert/encoder/layer_5/attention/self/MatMul_1, Execution Time: 0.001135 seconds Node: bert/encoder/layer_5/attention/self/transpose_3, Execution Time: 0.000409 seconds Node: bert/encoder/layer_5/attention/self/Reshape_3, Execution Time: 0.000035 seconds Node: bert/encoder/layer_5/attention/output/dense/MatMul, Execution Time: 0.000884 seconds Node: bert/encoder/layer_5/attention/output/dense/BiasAdd, Execution Time: 0.000612 seconds Node: bert/encoder/layer_5/attention/output/add, Execution Time: 0.000840 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/moments/mean, Execution Time: 0.000264 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000629 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/moments/SquaredDifference__379, Execution Time: 0.000816 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/moments/variance, Execution Time: 0.000245 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000079 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000057 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/Rsqrt__381, Execution Time: 0.000083 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000080 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000073 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000078 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000628 seconds Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000658 seconds Node: bert/encoder/layer_5/intermediate/dense/MatMul, Execution Time: 0.002167 seconds Node: bert/encoder/layer_5/intermediate/dense/BiasAdd, Execution Time: 0.001763 seconds Node: bert/encoder/layer_5/intermediate/dense/Pow, Execution Time: 0.001971 seconds Node: bert/encoder/layer_5/intermediate/dense/mul, Execution Time: 0.001911 seconds Node: bert/encoder/layer_5/intermediate/dense/add, Execution Time: 0.002553 seconds Node: bert/encoder/layer_5/intermediate/dense/mul_1, Execution Time: 0.001944 seconds Node: bert/encoder/layer_5/intermediate/dense/Tanh, Execution Time: 0.002001 seconds Node: bert/encoder/layer_5/intermediate/dense/add_1, Execution Time: 0.001930 seconds Node: bert/encoder/layer_5/intermediate/dense/mul_2, Execution Time: 0.001891 seconds Node: bert/encoder/layer_5/intermediate/dense/mul_3, Execution Time: 0.002441 seconds Node: bert/encoder/layer_5/output/dense/MatMul, Execution Time: 0.002590 seconds Node: bert/encoder/layer_5/output/dense/BiasAdd, Execution Time: 0.000355 seconds Node: bert/encoder/layer_5/output/add, Execution Time: 0.000619 seconds Node: bert/encoder/layer_5/output/LayerNorm/moments/mean, Execution Time: 0.000254 seconds Node: bert/encoder/layer_5/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000401 seconds Node: bert/encoder/layer_5/output/LayerNorm/moments/SquaredDifference__383, Execution Time: 0.000564 seconds Node: bert/encoder/layer_5/output/LayerNorm/moments/variance, Execution Time: 0.000243 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/add, Execution Time: 0.000083 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000058 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/Rsqrt__385, Execution Time: 0.000097 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/mul, Execution Time: 0.000077 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000079 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/sub, Execution Time: 0.000102 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000407 seconds Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000427 seconds Node: bert/encoder/layer_6/attention/self/value/MatMul, Execution Time: 0.000846 seconds Node: bert/encoder/layer_6/attention/self/value/BiasAdd, Execution Time: 0.000599 seconds Node: bert/encoder/layer_6/attention/self/Reshape_2, Execution Time: 0.000031 seconds Node: bert/encoder/layer_6/attention/self/transpose_2, Execution Time: 0.000653 seconds Node: bert/encoder/layer_6/attention/self/query/MatMul, Execution Time: 0.000833 seconds Node: bert/encoder/layer_6/attention/self/query/BiasAdd, Execution Time: 0.000587 seconds Node: bert/encoder/layer_6/attention/self/Reshape, Execution Time: 0.000017 seconds Node: bert/encoder/layer_6/attention/self/transpose, Execution Time: 0.000648 seconds Node: bert/encoder/layer_6/attention/self/key/MatMul, Execution Time: 0.000811 seconds Node: bert/encoder/layer_6/attention/self/key/BiasAdd, Execution Time: 0.000607 seconds Node: bert/encoder/layer_6/attention/self/Reshape_1, Execution Time: 0.000011 seconds Node: bert/encoder/layer_6/attention/self/MatMul__390, Execution Time: 0.000661 seconds Node: bert/encoder/layer_6/attention/self/MatMul, Execution Time: 0.001576 seconds Node: bert/encoder/layer_6/attention/self/Mul, Execution Time: 0.001204 seconds Node: bert/encoder/layer_6/attention/self/add, Execution Time: 0.001848 seconds Node: bert/encoder/layer_6/attention/self/Softmax, Execution Time: 0.001984 seconds Node: bert/encoder/layer_6/attention/self/MatMul_1, Execution Time: 0.001079 seconds Node: bert/encoder/layer_6/attention/self/transpose_3, Execution Time: 0.000350 seconds Node: bert/encoder/layer_6/attention/self/Reshape_3, Execution Time: 0.000041 seconds Node: bert/encoder/layer_6/attention/output/dense/MatMul, Execution Time: 0.000838 seconds Node: bert/encoder/layer_6/attention/output/dense/BiasAdd, Execution Time: 0.000611 seconds Node: bert/encoder/layer_6/attention/output/add, Execution Time: 0.000853 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/moments/mean, Execution Time: 0.000258 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000620 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/moments/SquaredDifference__393, Execution Time: 0.000802 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/moments/variance, Execution Time: 0.000266 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000080 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000061 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/Rsqrt__395, Execution Time: 0.000081 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000076 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000072 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000074 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000652 seconds Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000685 seconds Node: bert/encoder/layer_6/intermediate/dense/MatMul, Execution Time: 0.002253 seconds Node: bert/encoder/layer_6/intermediate/dense/BiasAdd, Execution Time: 0.001844 seconds Node: bert/encoder/layer_6/intermediate/dense/Pow, Execution Time: 0.002031 seconds Node: bert/encoder/layer_6/intermediate/dense/mul, Execution Time: 0.001997 seconds Node: bert/encoder/layer_6/intermediate/dense/add, Execution Time: 0.002475 seconds Node: bert/encoder/layer_6/intermediate/dense/mul_1, Execution Time: 0.002036 seconds Node: bert/encoder/layer_6/intermediate/dense/Tanh, Execution Time: 0.001874 seconds Node: bert/encoder/layer_6/intermediate/dense/add_1, Execution Time: 0.001909 seconds Node: bert/encoder/layer_6/intermediate/dense/mul_2, Execution Time: 0.001871 seconds Node: bert/encoder/layer_6/intermediate/dense/mul_3, Execution Time: 0.002447 seconds Node: bert/encoder/layer_6/output/dense/MatMul, Execution Time: 0.002618 seconds Node: bert/encoder/layer_6/output/dense/BiasAdd, Execution Time: 0.000363 seconds Node: bert/encoder/layer_6/output/add, Execution Time: 0.000618 seconds Node: bert/encoder/layer_6/output/LayerNorm/moments/mean, Execution Time: 0.000257 seconds Node: bert/encoder/layer_6/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000398 seconds Node: bert/encoder/layer_6/output/LayerNorm/moments/SquaredDifference__397, Execution Time: 0.000574 seconds Node: bert/encoder/layer_6/output/LayerNorm/moments/variance, Execution Time: 0.000236 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/add, Execution Time: 0.000077 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000056 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/Rsqrt__399, Execution Time: 0.000094 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/mul, Execution Time: 0.000091 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000098 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/sub, Execution Time: 0.000086 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000445 seconds Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000418 seconds Node: bert/encoder/layer_7/attention/self/value/MatMul, Execution Time: 0.000828 seconds Node: bert/encoder/layer_7/attention/self/value/BiasAdd, Execution Time: 0.000648 seconds Node: bert/encoder/layer_7/attention/self/Reshape_2, Execution Time: 0.000031 seconds Node: bert/encoder/layer_7/attention/self/transpose_2, Execution Time: 0.000654 seconds Node: bert/encoder/layer_7/attention/self/query/MatMul, Execution Time: 0.000823 seconds Node: bert/encoder/layer_7/attention/self/query/BiasAdd, Execution Time: 0.000593 seconds Node: bert/encoder/layer_7/attention/self/Reshape, Execution Time: 0.000012 seconds Node: bert/encoder/layer_7/attention/self/transpose, Execution Time: 0.000661 seconds Node: bert/encoder/layer_7/attention/self/key/MatMul, Execution Time: 0.000830 seconds Node: bert/encoder/layer_7/attention/self/key/BiasAdd, Execution Time: 0.000610 seconds Node: bert/encoder/layer_7/attention/self/Reshape_1, Execution Time: 0.000011 seconds Node: bert/encoder/layer_7/attention/self/MatMul__404, Execution Time: 0.000669 seconds Node: bert/encoder/layer_7/attention/self/MatMul, Execution Time: 0.001552 seconds Node: bert/encoder/layer_7/attention/self/Mul, Execution Time: 0.001162 seconds Node: bert/encoder/layer_7/attention/self/add, Execution Time: 0.001778 seconds Node: bert/encoder/layer_7/attention/self/Softmax, Execution Time: 0.001961 seconds Node: bert/encoder/layer_7/attention/self/MatMul_1, Execution Time: 0.001072 seconds Node: bert/encoder/layer_7/attention/self/transpose_3, Execution Time: 0.000353 seconds Node: bert/encoder/layer_7/attention/self/Reshape_3, Execution Time: 0.000044 seconds Node: bert/encoder/layer_7/attention/output/dense/MatMul, Execution Time: 0.000816 seconds Node: bert/encoder/layer_7/attention/output/dense/BiasAdd, Execution Time: 0.000583 seconds Node: bert/encoder/layer_7/attention/output/add, Execution Time: 0.000847 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/moments/mean, Execution Time: 0.000264 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000653 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/moments/SquaredDifference__407, Execution Time: 0.000865 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/moments/variance, Execution Time: 0.000246 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000082 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000057 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/Rsqrt__409, Execution Time: 0.000197 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000078 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000073 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000075 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000655 seconds Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000886 seconds Node: bert/encoder/layer_7/intermediate/dense/MatMul, Execution Time: 0.003348 seconds Node: bert/encoder/layer_7/intermediate/dense/BiasAdd, Execution Time: 0.002504 seconds Node: bert/encoder/layer_7/intermediate/dense/Pow, Execution Time: 0.002551 seconds Node: bert/encoder/layer_7/intermediate/dense/mul, Execution Time: 0.001902 seconds Node: bert/encoder/layer_7/intermediate/dense/add, Execution Time: 0.002387 seconds Node: bert/encoder/layer_7/intermediate/dense/mul_1, Execution Time: 0.001894 seconds Node: bert/encoder/layer_7/intermediate/dense/Tanh, Execution Time: 0.001869 seconds Node: bert/encoder/layer_7/intermediate/dense/add_1, Execution Time: 0.001886 seconds Node: bert/encoder/layer_7/intermediate/dense/mul_2, Execution Time: 0.001863 seconds Node: bert/encoder/layer_7/intermediate/dense/mul_3, Execution Time: 0.002483 seconds Node: bert/encoder/layer_7/output/dense/MatMul, Execution Time: 0.002611 seconds Node: bert/encoder/layer_7/output/dense/BiasAdd, Execution Time: 0.000362 seconds Node: bert/encoder/layer_7/output/add, Execution Time: 0.000606 seconds Node: bert/encoder/layer_7/output/LayerNorm/moments/mean, Execution Time: 0.000263 seconds Node: bert/encoder/layer_7/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000391 seconds Node: bert/encoder/layer_7/output/LayerNorm/moments/SquaredDifference__411, Execution Time: 0.000586 seconds Node: bert/encoder/layer_7/output/LayerNorm/moments/variance, Execution Time: 0.000242 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/add, Execution Time: 0.000073 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000057 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/Rsqrt__413, Execution Time: 0.000103 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/mul, Execution Time: 0.000078 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000073 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/sub, Execution Time: 0.000076 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000393 seconds Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000443 seconds Node: bert/encoder/layer_8/attention/self/value/MatMul, Execution Time: 0.000826 seconds Node: bert/encoder/layer_8/attention/self/value/BiasAdd, Execution Time: 0.000650 seconds Node: bert/encoder/layer_8/attention/self/Reshape_2, Execution Time: 0.000031 seconds Node: bert/encoder/layer_8/attention/self/transpose_2, Execution Time: 0.000665 seconds Node: bert/encoder/layer_8/attention/self/query/MatMul, Execution Time: 0.000834 seconds Node: bert/encoder/layer_8/attention/self/query/BiasAdd, Execution Time: 0.000612 seconds Node: bert/encoder/layer_8/attention/self/Reshape, Execution Time: 0.000014 seconds Node: bert/encoder/layer_8/attention/self/transpose, Execution Time: 0.000664 seconds Node: bert/encoder/layer_8/attention/self/key/MatMul, Execution Time: 0.000814 seconds Node: bert/encoder/layer_8/attention/self/key/BiasAdd, Execution Time: 0.000604 seconds Node: bert/encoder/layer_8/attention/self/Reshape_1, Execution Time: 0.000011 seconds Node: bert/encoder/layer_8/attention/self/MatMul__418, Execution Time: 0.000694 seconds Node: bert/encoder/layer_8/attention/self/MatMul, Execution Time: 0.001452 seconds Node: bert/encoder/layer_8/attention/self/Mul, Execution Time: 0.001154 seconds Node: bert/encoder/layer_8/attention/self/add, Execution Time: 0.001780 seconds Node: bert/encoder/layer_8/attention/self/Softmax, Execution Time: 0.001961 seconds Node: bert/encoder/layer_8/attention/self/MatMul_1, Execution Time: 0.001049 seconds Node: bert/encoder/layer_8/attention/self/transpose_3, Execution Time: 0.000341 seconds Node: bert/encoder/layer_8/attention/self/Reshape_3, Execution Time: 0.000036 seconds Node: bert/encoder/layer_8/attention/output/dense/MatMul, Execution Time: 0.000810 seconds Node: bert/encoder/layer_8/attention/output/dense/BiasAdd, Execution Time: 0.000597 seconds Node: bert/encoder/layer_8/attention/output/add, Execution Time: 0.000856 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/moments/mean, Execution Time: 0.000271 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000654 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/moments/SquaredDifference__421, Execution Time: 0.000886 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/moments/variance, Execution Time: 0.000243 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000078 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000057 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/Rsqrt__423, Execution Time: 0.000094 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000086 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000076 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000080 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000635 seconds Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000665 seconds Node: bert/encoder/layer_8/intermediate/dense/MatMul, Execution Time: 0.002090 seconds Node: bert/encoder/layer_8/intermediate/dense/BiasAdd, Execution Time: 0.001770 seconds Node: bert/encoder/layer_8/intermediate/dense/Pow, Execution Time: 0.001944 seconds Node: bert/encoder/layer_8/intermediate/dense/mul, Execution Time: 0.001942 seconds Node: bert/encoder/layer_8/intermediate/dense/add, Execution Time: 0.002400 seconds Node: bert/encoder/layer_8/intermediate/dense/mul_1, Execution Time: 0.001897 seconds Node: bert/encoder/layer_8/intermediate/dense/Tanh, Execution Time: 0.001899 seconds Node: bert/encoder/layer_8/intermediate/dense/add_1, Execution Time: 0.001938 seconds Node: bert/encoder/layer_8/intermediate/dense/mul_2, Execution Time: 0.001914 seconds Node: bert/encoder/layer_8/intermediate/dense/mul_3, Execution Time: 0.002474 seconds Node: bert/encoder/layer_8/output/dense/MatMul, Execution Time: 0.002604 seconds Node: bert/encoder/layer_8/output/dense/BiasAdd, Execution Time: 0.000363 seconds Node: bert/encoder/layer_8/output/add, Execution Time: 0.000621 seconds Node: bert/encoder/layer_8/output/LayerNorm/moments/mean, Execution Time: 0.000251 seconds Node: bert/encoder/layer_8/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000394 seconds Node: bert/encoder/layer_8/output/LayerNorm/moments/SquaredDifference__425, Execution Time: 0.000578 seconds Node: bert/encoder/layer_8/output/LayerNorm/moments/variance, Execution Time: 0.000236 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/add, Execution Time: 0.000080 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000063 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/Rsqrt__427, Execution Time: 0.000083 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/mul, Execution Time: 0.000076 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000072 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/sub, Execution Time: 0.000074 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000404 seconds Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000437 seconds Node: bert/encoder/layer_9/attention/self/value/MatMul, Execution Time: 0.000854 seconds Node: bert/encoder/layer_9/attention/self/value/BiasAdd, Execution Time: 0.000609 seconds Node: bert/encoder/layer_9/attention/self/Reshape_2, Execution Time: 0.000030 seconds Node: bert/encoder/layer_9/attention/self/transpose_2, Execution Time: 0.000660 seconds Node: bert/encoder/layer_9/attention/self/query/MatMul, Execution Time: 0.000815 seconds Node: bert/encoder/layer_9/attention/self/query/BiasAdd, Execution Time: 0.000588 seconds Node: bert/encoder/layer_9/attention/self/Reshape, Execution Time: 0.000012 seconds Node: bert/encoder/layer_9/attention/self/transpose, Execution Time: 0.000708 seconds Node: bert/encoder/layer_9/attention/self/key/MatMul, Execution Time: 0.000826 seconds Node: bert/encoder/layer_9/attention/self/key/BiasAdd, Execution Time: 0.000698 seconds Node: bert/encoder/layer_9/attention/self/Reshape_1, Execution Time: 0.000012 seconds Node: bert/encoder/layer_9/attention/self/MatMul__432, Execution Time: 0.000833 seconds Node: bert/encoder/layer_9/attention/self/MatMul, Execution Time: 0.001895 seconds Node: bert/encoder/layer_9/attention/self/Mul, Execution Time: 0.001525 seconds Node: bert/encoder/layer_9/attention/self/add, Execution Time: 0.002471 seconds Node: bert/encoder/layer_9/attention/self/Softmax, Execution Time: 0.002511 seconds Node: bert/encoder/layer_9/attention/self/MatMul_1, Execution Time: 0.001042 seconds Node: bert/encoder/layer_9/attention/self/transpose_3, Execution Time: 0.000381 seconds Node: bert/encoder/layer_9/attention/self/Reshape_3, Execution Time: 0.000038 seconds Node: bert/encoder/layer_9/attention/output/dense/MatMul, Execution Time: 0.000832 seconds Node: bert/encoder/layer_9/attention/output/dense/BiasAdd, Execution Time: 0.000627 seconds Node: bert/encoder/layer_9/attention/output/add, Execution Time: 0.000857 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/moments/mean, Execution Time: 0.000256 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000655 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/moments/SquaredDifference__435, Execution Time: 0.000810 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/moments/variance, Execution Time: 0.000259 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000089 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000057 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/Rsqrt__437, Execution Time: 0.000093 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000079 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000080 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000076 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000639 seconds Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000655 seconds Node: bert/encoder/layer_9/intermediate/dense/MatMul, Execution Time: 0.002046 seconds Node: bert/encoder/layer_9/intermediate/dense/BiasAdd, Execution Time: 0.001705 seconds Node: bert/encoder/layer_9/intermediate/dense/Pow, Execution Time: 0.001965 seconds Node: bert/encoder/layer_9/intermediate/dense/mul, Execution Time: 0.002001 seconds Node: bert/encoder/layer_9/intermediate/dense/add, Execution Time: 0.002467 seconds Node: bert/encoder/layer_9/intermediate/dense/mul_1, Execution Time: 0.001926 seconds Node: bert/encoder/layer_9/intermediate/dense/Tanh, Execution Time: 0.001907 seconds Node: bert/encoder/layer_9/intermediate/dense/add_1, Execution Time: 0.001983 seconds Node: bert/encoder/layer_9/intermediate/dense/mul_2, Execution Time: 0.001966 seconds Node: bert/encoder/layer_9/intermediate/dense/mul_3, Execution Time: 0.002498 seconds Node: bert/encoder/layer_9/output/dense/MatMul, Execution Time: 0.002375 seconds Node: bert/encoder/layer_9/output/dense/BiasAdd, Execution Time: 0.000355 seconds Node: bert/encoder/layer_9/output/add, Execution Time: 0.000615 seconds Node: bert/encoder/layer_9/output/LayerNorm/moments/mean, Execution Time: 0.000272 seconds Node: bert/encoder/layer_9/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000394 seconds Node: bert/encoder/layer_9/output/LayerNorm/moments/SquaredDifference__439, Execution Time: 0.000573 seconds Node: bert/encoder/layer_9/output/LayerNorm/moments/variance, Execution Time: 0.000257 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/add, Execution Time: 0.000079 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000057 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/Rsqrt__441, Execution Time: 0.000079 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/mul, Execution Time: 0.000076 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000074 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/sub, Execution Time: 0.000077 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000391 seconds Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000761 seconds Node: bert/encoder/layer_10/attention/self/value/MatMul, Execution Time: 0.000843 seconds Node: bert/encoder/layer_10/attention/self/value/BiasAdd, Execution Time: 0.000601 seconds Node: bert/encoder/layer_10/attention/self/Reshape_2, Execution Time: 0.000030 seconds Node: bert/encoder/layer_10/attention/self/transpose_2, Execution Time: 0.000674 seconds Node: bert/encoder/layer_10/attention/self/query/MatMul, Execution Time: 0.000809 seconds Node: bert/encoder/layer_10/attention/self/query/BiasAdd, Execution Time: 0.000582 seconds Node: bert/encoder/layer_10/attention/self/Reshape, Execution Time: 0.000012 seconds Node: bert/encoder/layer_10/attention/self/transpose, Execution Time: 0.000638 seconds Node: bert/encoder/layer_10/attention/self/key/MatMul, Execution Time: 0.000884 seconds Node: bert/encoder/layer_10/attention/self/key/BiasAdd, Execution Time: 0.000572 seconds Node: bert/encoder/layer_10/attention/self/Reshape_1, Execution Time: 0.000011 seconds Node: bert/encoder/layer_10/attention/self/MatMul__446, Execution Time: 0.000645 seconds Node: bert/encoder/layer_10/attention/self/MatMul, Execution Time: 0.001483 seconds Node: bert/encoder/layer_10/attention/self/Mul, Execution Time: 0.001150 seconds Node: bert/encoder/layer_10/attention/self/add, Execution Time: 0.001845 seconds Node: bert/encoder/layer_10/attention/self/Softmax, Execution Time: 0.002062 seconds Node: bert/encoder/layer_10/attention/self/MatMul_1, Execution Time: 0.001049 seconds Node: bert/encoder/layer_10/attention/self/transpose_3, Execution Time: 0.000372 seconds Node: bert/encoder/layer_10/attention/self/Reshape_3, Execution Time: 0.000038 seconds Node: bert/encoder/layer_10/attention/output/dense/MatMul, Execution Time: 0.000818 seconds Node: bert/encoder/layer_10/attention/output/dense/BiasAdd, Execution Time: 0.000593 seconds Node: bert/encoder/layer_10/attention/output/add, Execution Time: 0.000886 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/moments/mean, Execution Time: 0.000260 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000638 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/moments/SquaredDifference__449, Execution Time: 0.000805 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/moments/variance, Execution Time: 0.000242 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000077 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000057 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/Rsqrt__451, Execution Time: 0.000096 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000082 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000077 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000076 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000625 seconds Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000699 seconds Node: bert/encoder/layer_10/intermediate/dense/MatMul, Execution Time: 0.002171 seconds Node: bert/encoder/layer_10/intermediate/dense/BiasAdd, Execution Time: 0.001771 seconds Node: bert/encoder/layer_10/intermediate/dense/Pow, Execution Time: 0.001981 seconds Node: bert/encoder/layer_10/intermediate/dense/mul, Execution Time: 0.001985 seconds Node: bert/encoder/layer_10/intermediate/dense/add, Execution Time: 0.002463 seconds Node: bert/encoder/layer_10/intermediate/dense/mul_1, Execution Time: 0.001956 seconds Node: bert/encoder/layer_10/intermediate/dense/Tanh, Execution Time: 0.001906 seconds Node: bert/encoder/layer_10/intermediate/dense/add_1, Execution Time: 0.001924 seconds Node: bert/encoder/layer_10/intermediate/dense/mul_2, Execution Time: 0.001986 seconds Node: bert/encoder/layer_10/intermediate/dense/mul_3, Execution Time: 0.002471 seconds Node: bert/encoder/layer_10/output/dense/MatMul, Execution Time: 0.002593 seconds Node: bert/encoder/layer_10/output/dense/BiasAdd, Execution Time: 0.000363 seconds Node: bert/encoder/layer_10/output/add, Execution Time: 0.000613 seconds Node: bert/encoder/layer_10/output/LayerNorm/moments/mean, Execution Time: 0.000251 seconds Node: bert/encoder/layer_10/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000399 seconds Node: bert/encoder/layer_10/output/LayerNorm/moments/SquaredDifference__453, Execution Time: 0.000568 seconds Node: bert/encoder/layer_10/output/LayerNorm/moments/variance, Execution Time: 0.000252 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/add, Execution Time: 0.000083 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000057 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/Rsqrt__455, Execution Time: 0.000079 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/mul, Execution Time: 0.000076 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000074 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/sub, Execution Time: 0.000083 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000406 seconds Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000424 seconds Node: bert/encoder/layer_11/attention/self/value/MatMul, Execution Time: 0.000851 seconds Node: bert/encoder/layer_11/attention/self/value/BiasAdd, Execution Time: 0.000589 seconds Node: bert/encoder/layer_11/attention/self/Reshape_2, Execution Time: 0.000030 seconds Node: bert/encoder/layer_11/attention/self/transpose_2, Execution Time: 0.000661 seconds Node: bert/encoder/layer_11/attention/self/query/MatMul, Execution Time: 0.000845 seconds Node: bert/encoder/layer_11/attention/self/query/BiasAdd, Execution Time: 0.000574 seconds Node: bert/encoder/layer_11/attention/self/Reshape, Execution Time: 0.000016 seconds Node: bert/encoder/layer_11/attention/self/transpose, Execution Time: 0.000676 seconds Node: bert/encoder/layer_11/attention/self/key/MatMul, Execution Time: 0.000789 seconds Node: bert/encoder/layer_11/attention/self/key/BiasAdd, Execution Time: 0.000604 seconds Node: bert/encoder/layer_11/attention/self/Reshape_1, Execution Time: 0.000011 seconds Node: bert/encoder/layer_11/attention/self/MatMul__460, Execution Time: 0.000644 seconds Node: bert/encoder/layer_11/attention/self/MatMul, Execution Time: 0.001563 seconds Node: bert/encoder/layer_11/attention/self/Mul, Execution Time: 0.001224 seconds Node: bert/encoder/layer_11/attention/self/add, Execution Time: 0.001846 seconds Node: bert/encoder/layer_11/attention/self/Softmax, Execution Time: 0.001970 seconds Node: bert/encoder/layer_11/attention/self/MatMul_1, Execution Time: 0.001055 seconds Node: bert/encoder/layer_11/attention/self/transpose_3, Execution Time: 0.000369 seconds Node: bert/encoder/layer_11/attention/self/Reshape_3, Execution Time: 0.000038 seconds Node: bert/encoder/layer_11/attention/output/dense/MatMul, Execution Time: 0.000781 seconds Node: bert/encoder/layer_11/attention/output/dense/BiasAdd, Execution Time: 0.000608 seconds Node: bert/encoder/layer_11/attention/output/add, Execution Time: 0.000904 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/moments/mean, Execution Time: 0.000268 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000654 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/moments/SquaredDifference__463, Execution Time: 0.000847 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/moments/variance, Execution Time: 0.000247 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000079 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000058 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/Rsqrt__465, Execution Time: 0.000094 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000078 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000073 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000076 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000658 seconds Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000678 seconds Node: bert/encoder/layer_11/intermediate/dense/MatMul, Execution Time: 0.002049 seconds Node: bert/encoder/layer_11/intermediate/dense/BiasAdd, Execution Time: 0.001844 seconds Node: bert/encoder/layer_11/intermediate/dense/Pow, Execution Time: 0.002036 seconds Node: bert/encoder/layer_11/intermediate/dense/mul, Execution Time: 0.002035 seconds Node: bert/encoder/layer_11/intermediate/dense/add, Execution Time: 0.002480 seconds Node: bert/encoder/layer_11/intermediate/dense/mul_1, Execution Time: 0.001962 seconds Node: bert/encoder/layer_11/intermediate/dense/Tanh, Execution Time: 0.001882 seconds Node: bert/encoder/layer_11/intermediate/dense/add_1, Execution Time: 0.001832 seconds Node: bert/encoder/layer_11/intermediate/dense/mul_2, Execution Time: 0.001885 seconds Node: bert/encoder/layer_11/intermediate/dense/mul_3, Execution Time: 0.002437 seconds Node: bert/encoder/layer_11/output/dense/MatMul, Execution Time: 0.002596 seconds Node: bert/encoder/layer_11/output/dense/BiasAdd, Execution Time: 0.000354 seconds Node: bert/encoder/layer_11/output/add, Execution Time: 0.000615 seconds Node: bert/encoder/layer_11/output/LayerNorm/moments/mean, Execution Time: 0.000266 seconds Node: bert/encoder/layer_11/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000396 seconds Node: bert/encoder/layer_11/output/LayerNorm/moments/SquaredDifference__467, Execution Time: 0.000575 seconds Node: bert/encoder/layer_11/output/LayerNorm/moments/variance, Execution Time: 0.000248 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/add, Execution Time: 0.000077 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000057 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/Rsqrt__469, Execution Time: 0.000096 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/mul, Execution Time: 0.000081 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000079 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/sub, Execution Time: 0.000077 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000389 seconds Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000430 seconds Node: MatMul, Execution Time: 0.009495 seconds Node: BiasAdd, Execution Time: 0.000202 seconds Node: Reshape_1, Execution Time: 0.000033 seconds Node: transpose, Execution Time: 0.000081 seconds Node: unstack, Execution Time: 0.000058 seconds Node: unstack__490, Execution Time: 0.000005 seconds Node: unstack__488, Execution Time: 0.000006 seconds Total Execution Time: 0.747523 seconds Total Matmul Fuse Execution Time: 0.000000 seconds Total Matmul + Add Execution Time: 0.379279 seconds Execution complete.
Editor is loading...
Leave a Comment