Untitled

Node Execution Times:
Node: unique_ids_graph_outputs_Identity__10, Execution Time: 0.000005 seconds
Node: bert/encoder/Shape, Execution Time: 0.000004 seconds
Node: bert/encoder/Shape__12, Execution Time: 0.000008 seconds
Node: bert/encoder/strided_slice, Execution Time: 0.000063 seconds
Node: bert/encoder/strided_slice__16, Execution Time: 0.000005 seconds
Node: bert/encoder/strided_slice__17, Execution Time: 0.000004 seconds
Node: bert/encoder/ones/packed_Unsqueeze__18, Execution Time: 0.000010 seconds
Node: bert/encoder/ones/packed_Concat__21, Execution Time: 0.000010 seconds
Node: bert/encoder/ones__22, Execution Time: 0.000004 seconds
Node: bert/encoder/ones, Execution Time: 0.000010 seconds
Node: bert/encoder/Reshape, Execution Time: 0.000007 seconds
Node: bert/encoder/Cast, Execution Time: 0.000004 seconds
Node: bert/encoder/mul, Execution Time: 0.028348 seconds
Node: bert/encoder/layer_9/attention/self/ExpandDims, Execution Time: 0.000036 seconds
Node: bert/encoder/layer_9/attention/self/sub, Execution Time: 0.006671 seconds
Node: bert/encoder/layer_9/attention/self/mul_1, Execution Time: 0.000339 seconds
Node: bert/embeddings/Reshape_2, Execution Time: 0.000030 seconds
Node: bert/embeddings/Reshape, Execution Time: 0.000005 seconds
Node: bert/embeddings/GatherV2, Execution Time: 0.000321 seconds
Node: bert/embeddings/Reshape_1, Execution Time: 0.000007 seconds
Node: bert/embeddings/one_hot, Execution Time: 0.000060 seconds
Node: bert/embeddings/MatMul, Execution Time: 0.060043 seconds
Node: bert/embeddings/Reshape_3, Execution Time: 0.000039 seconds
Node: bert/embeddings/add, Execution Time: 0.002195 seconds
Node: bert/embeddings/add_1, Execution Time: 0.001019 seconds
Node: bert/embeddings/LayerNorm/moments/mean, Execution Time: 0.005257 seconds
Node: bert/embeddings/LayerNorm/moments/SquaredDifference, Execution Time: 0.000700 seconds
Node: bert/embeddings/LayerNorm/moments/SquaredDifference__72, Execution Time: 0.000931 seconds
Node: bert/embeddings/LayerNorm/moments/variance, Execution Time: 0.000269 seconds
Node: bert/embeddings/LayerNorm/batchnorm/add, Execution Time: 0.000085 seconds
Node: bert/embeddings/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.009831 seconds
Node: bert/embeddings/LayerNorm/batchnorm/Rsqrt__74, Execution Time: 0.005222 seconds
Node: bert/embeddings/LayerNorm/batchnorm/mul, Execution Time: 0.000118 seconds
Node: bert/embeddings/LayerNorm/batchnorm/mul_2, Execution Time: 0.000078 seconds
Node: bert/embeddings/LayerNorm/batchnorm/sub, Execution Time: 0.000089 seconds
Node: bert/embeddings/LayerNorm/batchnorm/mul_1, Execution Time: 0.000747 seconds
Node: bert/embeddings/LayerNorm/batchnorm/add_1, Execution Time: 0.000670 seconds
Node: bert/encoder/Reshape_1, Execution Time: 0.000036 seconds
Node: bert/encoder/layer_0/attention/self/value/MatMul, Execution Time: 0.001816 seconds
Node: bert/encoder/layer_0/attention/self/value/BiasAdd, Execution Time: 0.000721 seconds
Node: bert/encoder/layer_0/attention/self/Reshape_2, Execution Time: 0.000015 seconds
Node: bert/encoder/layer_0/attention/self/transpose_2, Execution Time: 0.000725 seconds
Node: bert/encoder/layer_0/attention/self/query/MatMul, Execution Time: 0.000951 seconds
Node: bert/encoder/layer_0/attention/self/query/BiasAdd, Execution Time: 0.000602 seconds
Node: bert/encoder/layer_0/attention/self/Reshape, Execution Time: 0.000012 seconds
Node: bert/encoder/layer_0/attention/self/transpose, Execution Time: 0.000677 seconds
Node: bert/encoder/layer_0/attention/self/key/MatMul, Execution Time: 0.000889 seconds
Node: bert/encoder/layer_0/attention/self/key/BiasAdd, Execution Time: 0.000610 seconds
Node: bert/encoder/layer_0/attention/self/Reshape_1, Execution Time: 0.000011 seconds
Node: bert/encoder/layer_0/attention/self/MatMul__306, Execution Time: 0.000672 seconds
Node: bert/encoder/layer_0/attention/self/MatMul, Execution Time: 0.006449 seconds
Node: bert/encoder/layer_0/attention/self/Mul, Execution Time: 0.001237 seconds
Node: bert/encoder/layer_0/attention/self/add, Execution Time: 0.001903 seconds
Node: bert/encoder/layer_0/attention/self/Softmax, Execution Time: 0.009270 seconds
Node: bert/encoder/layer_0/attention/self/MatMul_1, Execution Time: 0.001393 seconds
Node: bert/encoder/layer_0/attention/self/transpose_3, Execution Time: 0.000262 seconds
Node: bert/encoder/layer_0/attention/self/Reshape_3, Execution Time: 0.000173 seconds
Node: bert/encoder/layer_0/attention/output/dense/MatMul, Execution Time: 0.000973 seconds
Node: bert/encoder/layer_0/attention/output/dense/BiasAdd, Execution Time: 0.000374 seconds
Node: bert/encoder/layer_0/attention/output/add, Execution Time: 0.000933 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/moments/mean, Execution Time: 0.000272 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000689 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/moments/SquaredDifference__309, Execution Time: 0.000891 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/moments/variance, Execution Time: 0.000254 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000083 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000061 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/Rsqrt__311, Execution Time: 0.000087 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000082 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000096 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000081 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000660 seconds
Node: bert/encoder/layer_0/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000679 seconds
Node: bert/encoder/layer_0/intermediate/dense/MatMul, Execution Time: 0.002888 seconds
Node: bert/encoder/layer_0/intermediate/dense/BiasAdd, Execution Time: 0.001701 seconds
Node: bert/encoder/layer_0/intermediate/dense/Pow, Execution Time: 0.017504 seconds
Node: bert/encoder/layer_0/intermediate/dense/mul, Execution Time: 0.002082 seconds
Node: bert/encoder/layer_0/intermediate/dense/add, Execution Time: 0.002460 seconds
Node: bert/encoder/layer_0/intermediate/dense/mul_1, Execution Time: 0.001976 seconds
Node: bert/encoder/layer_0/intermediate/dense/Tanh, Execution Time: 0.004290 seconds
Node: bert/encoder/layer_0/intermediate/dense/add_1, Execution Time: 0.001913 seconds
Node: bert/encoder/layer_0/intermediate/dense/mul_2, Execution Time: 0.001953 seconds
Node: bert/encoder/layer_0/intermediate/dense/mul_3, Execution Time: 0.002519 seconds
Node: bert/encoder/layer_0/output/dense/MatMul, Execution Time: 0.002670 seconds
Node: bert/encoder/layer_0/output/dense/BiasAdd, Execution Time: 0.000359 seconds
Node: bert/encoder/layer_0/output/add, Execution Time: 0.000611 seconds
Node: bert/encoder/layer_0/output/LayerNorm/moments/mean, Execution Time: 0.000258 seconds
Node: bert/encoder/layer_0/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000420 seconds
Node: bert/encoder/layer_0/output/LayerNorm/moments/SquaredDifference__313, Execution Time: 0.000582 seconds
Node: bert/encoder/layer_0/output/LayerNorm/moments/variance, Execution Time: 0.000250 seconds
Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/add, Execution Time: 0.000187 seconds
Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000064 seconds
Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/Rsqrt__315, Execution Time: 0.000083 seconds
Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/mul, Execution Time: 0.000081 seconds
Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000082 seconds
Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/sub, Execution Time: 0.000079 seconds
Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000401 seconds
Node: bert/encoder/layer_0/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000674 seconds
Node: bert/encoder/layer_1/attention/self/value/MatMul, Execution Time: 0.000956 seconds
Node: bert/encoder/layer_1/attention/self/value/BiasAdd, Execution Time: 0.000637 seconds
Node: bert/encoder/layer_1/attention/self/Reshape_2, Execution Time: 0.000030 seconds
Node: bert/encoder/layer_1/attention/self/transpose_2, Execution Time: 0.000665 seconds
Node: bert/encoder/layer_1/attention/self/query/MatMul, Execution Time: 0.000844 seconds
Node: bert/encoder/layer_1/attention/self/query/BiasAdd, Execution Time: 0.000628 seconds
Node: bert/encoder/layer_1/attention/self/Reshape, Execution Time: 0.000013 seconds
Node: bert/encoder/layer_1/attention/self/transpose, Execution Time: 0.000673 seconds
Node: bert/encoder/layer_1/attention/self/key/MatMul, Execution Time: 0.000858 seconds
Node: bert/encoder/layer_1/attention/self/key/BiasAdd, Execution Time: 0.000661 seconds
Node: bert/encoder/layer_1/attention/self/Reshape_1, Execution Time: 0.000012 seconds
Node: bert/encoder/layer_1/attention/self/MatMul__320, Execution Time: 0.000654 seconds
Node: bert/encoder/layer_1/attention/self/MatMul, Execution Time: 0.001526 seconds
Node: bert/encoder/layer_1/attention/self/Mul, Execution Time: 0.001167 seconds
Node: bert/encoder/layer_1/attention/self/add, Execution Time: 0.001798 seconds
Node: bert/encoder/layer_1/attention/self/Softmax, Execution Time: 0.002015 seconds
Node: bert/encoder/layer_1/attention/self/MatMul_1, Execution Time: 0.001103 seconds
Node: bert/encoder/layer_1/attention/self/transpose_3, Execution Time: 0.000239 seconds
Node: bert/encoder/layer_1/attention/self/Reshape_3, Execution Time: 0.000045 seconds
Node: bert/encoder/layer_1/attention/output/dense/MatMul, Execution Time: 0.000865 seconds
Node: bert/encoder/layer_1/attention/output/dense/BiasAdd, Execution Time: 0.000354 seconds
Node: bert/encoder/layer_1/attention/output/add, Execution Time: 0.000867 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/moments/mean, Execution Time: 0.000270 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000647 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/moments/SquaredDifference__323, Execution Time: 0.000906 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/moments/variance, Execution Time: 0.000246 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000084 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000066 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/Rsqrt__325, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000075 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000071 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000670 seconds
Node: bert/encoder/layer_1/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000688 seconds
Node: bert/encoder/layer_1/intermediate/dense/MatMul, Execution Time: 0.002165 seconds
Node: bert/encoder/layer_1/intermediate/dense/BiasAdd, Execution Time: 0.001831 seconds
Node: bert/encoder/layer_1/intermediate/dense/Pow, Execution Time: 0.001958 seconds
Node: bert/encoder/layer_1/intermediate/dense/mul, Execution Time: 0.001920 seconds
Node: bert/encoder/layer_1/intermediate/dense/add, Execution Time: 0.002380 seconds
Node: bert/encoder/layer_1/intermediate/dense/mul_1, Execution Time: 0.001852 seconds
Node: bert/encoder/layer_1/intermediate/dense/Tanh, Execution Time: 0.001966 seconds
Node: bert/encoder/layer_1/intermediate/dense/add_1, Execution Time: 0.002024 seconds
Node: bert/encoder/layer_1/intermediate/dense/mul_2, Execution Time: 0.002743 seconds
Node: bert/encoder/layer_1/intermediate/dense/mul_3, Execution Time: 0.003551 seconds
Node: bert/encoder/layer_1/output/dense/MatMul, Execution Time: 0.003203 seconds
Node: bert/encoder/layer_1/output/dense/BiasAdd, Execution Time: 0.000364 seconds
Node: bert/encoder/layer_1/output/add, Execution Time: 0.000639 seconds
Node: bert/encoder/layer_1/output/LayerNorm/moments/mean, Execution Time: 0.000268 seconds
Node: bert/encoder/layer_1/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000399 seconds
Node: bert/encoder/layer_1/output/LayerNorm/moments/SquaredDifference__327, Execution Time: 0.000582 seconds
Node: bert/encoder/layer_1/output/LayerNorm/moments/variance, Execution Time: 0.000250 seconds
Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/add, Execution Time: 0.000081 seconds
Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds
Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/Rsqrt__329, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/mul, Execution Time: 0.000085 seconds
Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000077 seconds
Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/sub, Execution Time: 0.000078 seconds
Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000402 seconds
Node: bert/encoder/layer_1/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000420 seconds
Node: bert/encoder/layer_2/attention/self/value/MatMul, Execution Time: 0.000864 seconds
Node: bert/encoder/layer_2/attention/self/value/BiasAdd, Execution Time: 0.000648 seconds
Node: bert/encoder/layer_2/attention/self/Reshape_2, Execution Time: 0.000030 seconds
Node: bert/encoder/layer_2/attention/self/transpose_2, Execution Time: 0.000665 seconds
Node: bert/encoder/layer_2/attention/self/query/MatMul, Execution Time: 0.000863 seconds
Node: bert/encoder/layer_2/attention/self/query/BiasAdd, Execution Time: 0.000624 seconds
Node: bert/encoder/layer_2/attention/self/Reshape, Execution Time: 0.000014 seconds
Node: bert/encoder/layer_2/attention/self/transpose, Execution Time: 0.000661 seconds
Node: bert/encoder/layer_2/attention/self/key/MatMul, Execution Time: 0.000831 seconds
Node: bert/encoder/layer_2/attention/self/key/BiasAdd, Execution Time: 0.000633 seconds
Node: bert/encoder/layer_2/attention/self/Reshape_1, Execution Time: 0.000012 seconds
Node: bert/encoder/layer_2/attention/self/MatMul__334, Execution Time: 0.000643 seconds
Node: bert/encoder/layer_2/attention/self/MatMul, Execution Time: 0.001513 seconds
Node: bert/encoder/layer_2/attention/self/Mul, Execution Time: 0.001163 seconds
Node: bert/encoder/layer_2/attention/self/add, Execution Time: 0.001790 seconds
Node: bert/encoder/layer_2/attention/self/Softmax, Execution Time: 0.002027 seconds
Node: bert/encoder/layer_2/attention/self/MatMul_1, Execution Time: 0.001088 seconds
Node: bert/encoder/layer_2/attention/self/transpose_3, Execution Time: 0.000241 seconds
Node: bert/encoder/layer_2/attention/self/Reshape_3, Execution Time: 0.000039 seconds
Node: bert/encoder/layer_2/attention/output/dense/MatMul, Execution Time: 0.000849 seconds
Node: bert/encoder/layer_2/attention/output/dense/BiasAdd, Execution Time: 0.000352 seconds
Node: bert/encoder/layer_2/attention/output/add, Execution Time: 0.000874 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/moments/mean, Execution Time: 0.000258 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000673 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/moments/SquaredDifference__337, Execution Time: 0.000888 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/moments/variance, Execution Time: 0.000270 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000087 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/Rsqrt__339, Execution Time: 0.000097 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000081 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000076 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000078 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000659 seconds
Node: bert/encoder/layer_2/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000694 seconds
Node: bert/encoder/layer_2/intermediate/dense/MatMul, Execution Time: 0.002074 seconds
Node: bert/encoder/layer_2/intermediate/dense/BiasAdd, Execution Time: 0.001761 seconds
Node: bert/encoder/layer_2/intermediate/dense/Pow, Execution Time: 0.001952 seconds
Node: bert/encoder/layer_2/intermediate/dense/mul, Execution Time: 0.001958 seconds
Node: bert/encoder/layer_2/intermediate/dense/add, Execution Time: 0.002382 seconds
Node: bert/encoder/layer_2/intermediate/dense/mul_1, Execution Time: 0.001869 seconds
Node: bert/encoder/layer_2/intermediate/dense/Tanh, Execution Time: 0.001868 seconds
Node: bert/encoder/layer_2/intermediate/dense/add_1, Execution Time: 0.001949 seconds
Node: bert/encoder/layer_2/intermediate/dense/mul_2, Execution Time: 0.001921 seconds
Node: bert/encoder/layer_2/intermediate/dense/mul_3, Execution Time: 0.002436 seconds
Node: bert/encoder/layer_2/output/dense/MatMul, Execution Time: 0.002524 seconds
Node: bert/encoder/layer_2/output/dense/BiasAdd, Execution Time: 0.000363 seconds
Node: bert/encoder/layer_2/output/add, Execution Time: 0.000603 seconds
Node: bert/encoder/layer_2/output/LayerNorm/moments/mean, Execution Time: 0.000265 seconds
Node: bert/encoder/layer_2/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000396 seconds
Node: bert/encoder/layer_2/output/LayerNorm/moments/SquaredDifference__341, Execution Time: 0.000578 seconds
Node: bert/encoder/layer_2/output/LayerNorm/moments/variance, Execution Time: 0.000249 seconds
Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/add, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000064 seconds
Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/Rsqrt__343, Execution Time: 0.000083 seconds
Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/mul, Execution Time: 0.000079 seconds
Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000075 seconds
Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/sub, Execution Time: 0.000078 seconds
Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000391 seconds
Node: bert/encoder/layer_2/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000434 seconds
Node: bert/encoder/layer_3/attention/self/value/MatMul, Execution Time: 0.000872 seconds
Node: bert/encoder/layer_3/attention/self/value/BiasAdd, Execution Time: 0.000619 seconds
Node: bert/encoder/layer_3/attention/self/Reshape_2, Execution Time: 0.000030 seconds
Node: bert/encoder/layer_3/attention/self/transpose_2, Execution Time: 0.000676 seconds
Node: bert/encoder/layer_3/attention/self/query/MatMul, Execution Time: 0.000849 seconds
Node: bert/encoder/layer_3/attention/self/query/BiasAdd, Execution Time: 0.000629 seconds
Node: bert/encoder/layer_3/attention/self/Reshape, Execution Time: 0.000013 seconds
Node: bert/encoder/layer_3/attention/self/transpose, Execution Time: 0.000712 seconds
Node: bert/encoder/layer_3/attention/self/key/MatMul, Execution Time: 0.000857 seconds
Node: bert/encoder/layer_3/attention/self/key/BiasAdd, Execution Time: 0.000601 seconds
Node: bert/encoder/layer_3/attention/self/Reshape_1, Execution Time: 0.000011 seconds
Node: bert/encoder/layer_3/attention/self/MatMul__348, Execution Time: 0.000675 seconds
Node: bert/encoder/layer_3/attention/self/MatMul, Execution Time: 0.001530 seconds
Node: bert/encoder/layer_3/attention/self/Mul, Execution Time: 0.001177 seconds
Node: bert/encoder/layer_3/attention/self/add, Execution Time: 0.001811 seconds
Node: bert/encoder/layer_3/attention/self/Softmax, Execution Time: 0.002022 seconds
Node: bert/encoder/layer_3/attention/self/MatMul_1, Execution Time: 0.001083 seconds
Node: bert/encoder/layer_3/attention/self/transpose_3, Execution Time: 0.000235 seconds
Node: bert/encoder/layer_3/attention/self/Reshape_3, Execution Time: 0.000040 seconds
Node: bert/encoder/layer_3/attention/output/dense/MatMul, Execution Time: 0.000864 seconds
Node: bert/encoder/layer_3/attention/output/dense/BiasAdd, Execution Time: 0.000349 seconds
Node: bert/encoder/layer_3/attention/output/add, Execution Time: 0.000879 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/moments/mean, Execution Time: 0.000267 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000653 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/moments/SquaredDifference__351, Execution Time: 0.000853 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/moments/variance, Execution Time: 0.000246 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000082 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000061 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/Rsqrt__353, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000082 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000077 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000083 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000652 seconds
Node: bert/encoder/layer_3/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000693 seconds
Node: bert/encoder/layer_3/intermediate/dense/MatMul, Execution Time: 0.002202 seconds
Node: bert/encoder/layer_3/intermediate/dense/BiasAdd, Execution Time: 0.001738 seconds
Node: bert/encoder/layer_3/intermediate/dense/Pow, Execution Time: 0.001960 seconds
Node: bert/encoder/layer_3/intermediate/dense/mul, Execution Time: 0.001963 seconds
Node: bert/encoder/layer_3/intermediate/dense/add, Execution Time: 0.002383 seconds
Node: bert/encoder/layer_3/intermediate/dense/mul_1, Execution Time: 0.001883 seconds
Node: bert/encoder/layer_3/intermediate/dense/Tanh, Execution Time: 0.001877 seconds
Node: bert/encoder/layer_3/intermediate/dense/add_1, Execution Time: 0.001869 seconds
Node: bert/encoder/layer_3/intermediate/dense/mul_2, Execution Time: 0.001913 seconds
Node: bert/encoder/layer_3/intermediate/dense/mul_3, Execution Time: 0.003111 seconds
Node: bert/encoder/layer_3/output/dense/MatMul, Execution Time: 0.003718 seconds
Node: bert/encoder/layer_3/output/dense/BiasAdd, Execution Time: 0.000514 seconds
Node: bert/encoder/layer_3/output/add, Execution Time: 0.000869 seconds
Node: bert/encoder/layer_3/output/LayerNorm/moments/mean, Execution Time: 0.000361 seconds
Node: bert/encoder/layer_3/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000470 seconds
Node: bert/encoder/layer_3/output/LayerNorm/moments/SquaredDifference__355, Execution Time: 0.000588 seconds
Node: bert/encoder/layer_3/output/LayerNorm/moments/variance, Execution Time: 0.000257 seconds
Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/add, Execution Time: 0.000089 seconds
Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000066 seconds
Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/Rsqrt__357, Execution Time: 0.000081 seconds
Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/mul, Execution Time: 0.000081 seconds
Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000077 seconds
Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/sub, Execution Time: 0.000079 seconds
Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000392 seconds
Node: bert/encoder/layer_3/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000440 seconds
Node: bert/encoder/layer_4/attention/self/value/MatMul, Execution Time: 0.000953 seconds
Node: bert/encoder/layer_4/attention/self/value/BiasAdd, Execution Time: 0.000351 seconds
Node: bert/encoder/layer_4/attention/self/Reshape_2, Execution Time: 0.000031 seconds
Node: bert/encoder/layer_4/attention/self/transpose_2, Execution Time: 0.000646 seconds
Node: bert/encoder/layer_4/attention/self/query/MatMul, Execution Time: 0.000894 seconds
Node: bert/encoder/layer_4/attention/self/query/BiasAdd, Execution Time: 0.000603 seconds
Node: bert/encoder/layer_4/attention/self/Reshape, Execution Time: 0.000014 seconds
Node: bert/encoder/layer_4/attention/self/transpose, Execution Time: 0.000654 seconds
Node: bert/encoder/layer_4/attention/self/key/MatMul, Execution Time: 0.000877 seconds
Node: bert/encoder/layer_4/attention/self/key/BiasAdd, Execution Time: 0.000592 seconds
Node: bert/encoder/layer_4/attention/self/Reshape_1, Execution Time: 0.000015 seconds
Node: bert/encoder/layer_4/attention/self/MatMul__362, Execution Time: 0.000640 seconds
Node: bert/encoder/layer_4/attention/self/MatMul, Execution Time: 0.001548 seconds
Node: bert/encoder/layer_4/attention/self/Mul, Execution Time: 0.001169 seconds
Node: bert/encoder/layer_4/attention/self/add, Execution Time: 0.001798 seconds
Node: bert/encoder/layer_4/attention/self/Softmax, Execution Time: 0.002026 seconds
Node: bert/encoder/layer_4/attention/self/MatMul_1, Execution Time: 0.001078 seconds
Node: bert/encoder/layer_4/attention/self/transpose_3, Execution Time: 0.000238 seconds
Node: bert/encoder/layer_4/attention/self/Reshape_3, Execution Time: 0.000040 seconds
Node: bert/encoder/layer_4/attention/output/dense/MatMul, Execution Time: 0.000826 seconds
Node: bert/encoder/layer_4/attention/output/dense/BiasAdd, Execution Time: 0.000348 seconds
Node: bert/encoder/layer_4/attention/output/add, Execution Time: 0.000884 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/moments/mean, Execution Time: 0.000262 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000673 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/moments/SquaredDifference__365, Execution Time: 0.000831 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/moments/variance, Execution Time: 0.000249 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000086 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000061 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/Rsqrt__367, Execution Time: 0.000095 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000086 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000087 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000079 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000638 seconds
Node: bert/encoder/layer_4/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000686 seconds
Node: bert/encoder/layer_4/intermediate/dense/MatMul, Execution Time: 0.002161 seconds
Node: bert/encoder/layer_4/intermediate/dense/BiasAdd, Execution Time: 0.001739 seconds
Node: bert/encoder/layer_4/intermediate/dense/Pow, Execution Time: 0.001952 seconds
Node: bert/encoder/layer_4/intermediate/dense/mul, Execution Time: 0.001911 seconds
Node: bert/encoder/layer_4/intermediate/dense/add, Execution Time: 0.002425 seconds
Node: bert/encoder/layer_4/intermediate/dense/mul_1, Execution Time: 0.001887 seconds
Node: bert/encoder/layer_4/intermediate/dense/Tanh, Execution Time: 0.001851 seconds
Node: bert/encoder/layer_4/intermediate/dense/add_1, Execution Time: 0.001900 seconds
Node: bert/encoder/layer_4/intermediate/dense/mul_2, Execution Time: 0.001927 seconds
Node: bert/encoder/layer_4/intermediate/dense/mul_3, Execution Time: 0.002463 seconds
Node: bert/encoder/layer_4/output/dense/MatMul, Execution Time: 0.002565 seconds
Node: bert/encoder/layer_4/output/dense/BiasAdd, Execution Time: 0.000361 seconds
Node: bert/encoder/layer_4/output/add, Execution Time: 0.000633 seconds
Node: bert/encoder/layer_4/output/LayerNorm/moments/mean, Execution Time: 0.000260 seconds
Node: bert/encoder/layer_4/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000414 seconds
Node: bert/encoder/layer_4/output/LayerNorm/moments/SquaredDifference__369, Execution Time: 0.000572 seconds
Node: bert/encoder/layer_4/output/LayerNorm/moments/variance, Execution Time: 0.000257 seconds
Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/add, Execution Time: 0.000081 seconds
Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000073 seconds
Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/Rsqrt__371, Execution Time: 0.000088 seconds
Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/mul, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000076 seconds
Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/sub, Execution Time: 0.000077 seconds
Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000389 seconds
Node: bert/encoder/layer_4/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000434 seconds
Node: bert/encoder/layer_5/attention/self/value/MatMul, Execution Time: 0.000885 seconds
Node: bert/encoder/layer_5/attention/self/value/BiasAdd, Execution Time: 0.000345 seconds
Node: bert/encoder/layer_5/attention/self/Reshape_2, Execution Time: 0.000032 seconds
Node: bert/encoder/layer_5/attention/self/transpose_2, Execution Time: 0.000682 seconds
Node: bert/encoder/layer_5/attention/self/query/MatMul, Execution Time: 0.000862 seconds
Node: bert/encoder/layer_5/attention/self/query/BiasAdd, Execution Time: 0.000638 seconds
Node: bert/encoder/layer_5/attention/self/Reshape, Execution Time: 0.000014 seconds
Node: bert/encoder/layer_5/attention/self/transpose, Execution Time: 0.000647 seconds
Node: bert/encoder/layer_5/attention/self/key/MatMul, Execution Time: 0.000824 seconds
Node: bert/encoder/layer_5/attention/self/key/BiasAdd, Execution Time: 0.000592 seconds
Node: bert/encoder/layer_5/attention/self/Reshape_1, Execution Time: 0.000011 seconds
Node: bert/encoder/layer_5/attention/self/MatMul__376, Execution Time: 0.000640 seconds
Node: bert/encoder/layer_5/attention/self/MatMul, Execution Time: 0.001573 seconds
Node: bert/encoder/layer_5/attention/self/Mul, Execution Time: 0.001202 seconds
Node: bert/encoder/layer_5/attention/self/add, Execution Time: 0.001837 seconds
Node: bert/encoder/layer_5/attention/self/Softmax, Execution Time: 0.001971 seconds
Node: bert/encoder/layer_5/attention/self/MatMul_1, Execution Time: 0.001069 seconds
Node: bert/encoder/layer_5/attention/self/transpose_3, Execution Time: 0.000239 seconds
Node: bert/encoder/layer_5/attention/self/Reshape_3, Execution Time: 0.000037 seconds
Node: bert/encoder/layer_5/attention/output/dense/MatMul, Execution Time: 0.000876 seconds
Node: bert/encoder/layer_5/attention/output/dense/BiasAdd, Execution Time: 0.000349 seconds
Node: bert/encoder/layer_5/attention/output/add, Execution Time: 0.000932 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/moments/mean, Execution Time: 0.000270 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000683 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/moments/SquaredDifference__379, Execution Time: 0.000845 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/moments/variance, Execution Time: 0.000249 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000082 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/Rsqrt__381, Execution Time: 0.000084 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000087 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000077 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000088 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000658 seconds
Node: bert/encoder/layer_5/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000703 seconds
Node: bert/encoder/layer_5/intermediate/dense/MatMul, Execution Time: 0.002141 seconds
Node: bert/encoder/layer_5/intermediate/dense/BiasAdd, Execution Time: 0.001723 seconds
Node: bert/encoder/layer_5/intermediate/dense/Pow, Execution Time: 0.002041 seconds
Node: bert/encoder/layer_5/intermediate/dense/mul, Execution Time: 0.001913 seconds
Node: bert/encoder/layer_5/intermediate/dense/add, Execution Time: 0.002433 seconds
Node: bert/encoder/layer_5/intermediate/dense/mul_1, Execution Time: 0.001894 seconds
Node: bert/encoder/layer_5/intermediate/dense/Tanh, Execution Time: 0.001852 seconds
Node: bert/encoder/layer_5/intermediate/dense/add_1, Execution Time: 0.001902 seconds
Node: bert/encoder/layer_5/intermediate/dense/mul_2, Execution Time: 0.001873 seconds
Node: bert/encoder/layer_5/intermediate/dense/mul_3, Execution Time: 0.002423 seconds
Node: bert/encoder/layer_5/output/dense/MatMul, Execution Time: 0.002789 seconds
Node: bert/encoder/layer_5/output/dense/BiasAdd, Execution Time: 0.000363 seconds
Node: bert/encoder/layer_5/output/add, Execution Time: 0.000637 seconds
Node: bert/encoder/layer_5/output/LayerNorm/moments/mean, Execution Time: 0.000260 seconds
Node: bert/encoder/layer_5/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000403 seconds
Node: bert/encoder/layer_5/output/LayerNorm/moments/SquaredDifference__383, Execution Time: 0.000575 seconds
Node: bert/encoder/layer_5/output/LayerNorm/moments/variance, Execution Time: 0.000249 seconds
Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/add, Execution Time: 0.000084 seconds
Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000059 seconds
Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/Rsqrt__385, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/mul, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000075 seconds
Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/sub, Execution Time: 0.000087 seconds
Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000388 seconds
Node: bert/encoder/layer_5/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000431 seconds
Node: bert/encoder/layer_6/attention/self/value/MatMul, Execution Time: 0.000896 seconds
Node: bert/encoder/layer_6/attention/self/value/BiasAdd, Execution Time: 0.000350 seconds
Node: bert/encoder/layer_6/attention/self/Reshape_2, Execution Time: 0.000031 seconds
Node: bert/encoder/layer_6/attention/self/transpose_2, Execution Time: 0.000684 seconds
Node: bert/encoder/layer_6/attention/self/query/MatMul, Execution Time: 0.000906 seconds
Node: bert/encoder/layer_6/attention/self/query/BiasAdd, Execution Time: 0.000597 seconds
Node: bert/encoder/layer_6/attention/self/Reshape, Execution Time: 0.000013 seconds
Node: bert/encoder/layer_6/attention/self/transpose, Execution Time: 0.000652 seconds
Node: bert/encoder/layer_6/attention/self/key/MatMul, Execution Time: 0.000835 seconds
Node: bert/encoder/layer_6/attention/self/key/BiasAdd, Execution Time: 0.000590 seconds
Node: bert/encoder/layer_6/attention/self/Reshape_1, Execution Time: 0.000011 seconds
Node: bert/encoder/layer_6/attention/self/MatMul__390, Execution Time: 0.000656 seconds
Node: bert/encoder/layer_6/attention/self/MatMul, Execution Time: 0.001508 seconds
Node: bert/encoder/layer_6/attention/self/Mul, Execution Time: 0.001217 seconds
Node: bert/encoder/layer_6/attention/self/add, Execution Time: 0.001867 seconds
Node: bert/encoder/layer_6/attention/self/Softmax, Execution Time: 0.001996 seconds
Node: bert/encoder/layer_6/attention/self/MatMul_1, Execution Time: 0.001116 seconds
Node: bert/encoder/layer_6/attention/self/transpose_3, Execution Time: 0.000239 seconds
Node: bert/encoder/layer_6/attention/self/Reshape_3, Execution Time: 0.000037 seconds
Node: bert/encoder/layer_6/attention/output/dense/MatMul, Execution Time: 0.000836 seconds
Node: bert/encoder/layer_6/attention/output/dense/BiasAdd, Execution Time: 0.000347 seconds
Node: bert/encoder/layer_6/attention/output/add, Execution Time: 0.000882 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/moments/mean, Execution Time: 0.000269 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000690 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/moments/SquaredDifference__393, Execution Time: 0.000840 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/moments/variance, Execution Time: 0.000247 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000083 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000062 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/Rsqrt__395, Execution Time: 0.000081 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000088 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000082 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000079 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000676 seconds
Node: bert/encoder/layer_6/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000665 seconds
Node: bert/encoder/layer_6/intermediate/dense/MatMul, Execution Time: 0.002177 seconds
Node: bert/encoder/layer_6/intermediate/dense/BiasAdd, Execution Time: 0.001727 seconds
Node: bert/encoder/layer_6/intermediate/dense/Pow, Execution Time: 0.002012 seconds
Node: bert/encoder/layer_6/intermediate/dense/mul, Execution Time: 0.001926 seconds
Node: bert/encoder/layer_6/intermediate/dense/add, Execution Time: 0.002371 seconds
Node: bert/encoder/layer_6/intermediate/dense/mul_1, Execution Time: 0.001931 seconds
Node: bert/encoder/layer_6/intermediate/dense/Tanh, Execution Time: 0.001858 seconds
Node: bert/encoder/layer_6/intermediate/dense/add_1, Execution Time: 0.002290 seconds
Node: bert/encoder/layer_6/intermediate/dense/mul_2, Execution Time: 0.002000 seconds
Node: bert/encoder/layer_6/intermediate/dense/mul_3, Execution Time: 0.002535 seconds
Node: bert/encoder/layer_6/output/dense/MatMul, Execution Time: 0.002536 seconds
Node: bert/encoder/layer_6/output/dense/BiasAdd, Execution Time: 0.000397 seconds
Node: bert/encoder/layer_6/output/add, Execution Time: 0.000596 seconds
Node: bert/encoder/layer_6/output/LayerNorm/moments/mean, Execution Time: 0.000259 seconds
Node: bert/encoder/layer_6/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000400 seconds
Node: bert/encoder/layer_6/output/LayerNorm/moments/SquaredDifference__397, Execution Time: 0.000589 seconds
Node: bert/encoder/layer_6/output/LayerNorm/moments/variance, Execution Time: 0.000244 seconds
Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/add, Execution Time: 0.000081 seconds
Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000059 seconds
Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/Rsqrt__399, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/mul, Execution Time: 0.000083 seconds
Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000078 seconds
Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/sub, Execution Time: 0.000078 seconds
Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000454 seconds
Node: bert/encoder/layer_6/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000433 seconds
Node: bert/encoder/layer_7/attention/self/value/MatMul, Execution Time: 0.000878 seconds
Node: bert/encoder/layer_7/attention/self/value/BiasAdd, Execution Time: 0.000659 seconds
Node: bert/encoder/layer_7/attention/self/Reshape_2, Execution Time: 0.000031 seconds
Node: bert/encoder/layer_7/attention/self/transpose_2, Execution Time: 0.000688 seconds
Node: bert/encoder/layer_7/attention/self/query/MatMul, Execution Time: 0.000833 seconds
Node: bert/encoder/layer_7/attention/self/query/BiasAdd, Execution Time: 0.000600 seconds
Node: bert/encoder/layer_7/attention/self/Reshape, Execution Time: 0.000014 seconds
Node: bert/encoder/layer_7/attention/self/transpose, Execution Time: 0.000653 seconds
Node: bert/encoder/layer_7/attention/self/key/MatMul, Execution Time: 0.000842 seconds
Node: bert/encoder/layer_7/attention/self/key/BiasAdd, Execution Time: 0.000603 seconds
Node: bert/encoder/layer_7/attention/self/Reshape_1, Execution Time: 0.000012 seconds
Node: bert/encoder/layer_7/attention/self/MatMul__404, Execution Time: 0.000651 seconds
Node: bert/encoder/layer_7/attention/self/MatMul, Execution Time: 0.001516 seconds
Node: bert/encoder/layer_7/attention/self/Mul, Execution Time: 0.001172 seconds
Node: bert/encoder/layer_7/attention/self/add, Execution Time: 0.001859 seconds
Node: bert/encoder/layer_7/attention/self/Softmax, Execution Time: 0.002170 seconds
Node: bert/encoder/layer_7/attention/self/MatMul_1, Execution Time: 0.001101 seconds
Node: bert/encoder/layer_7/attention/self/transpose_3, Execution Time: 0.000285 seconds
Node: bert/encoder/layer_7/attention/self/Reshape_3, Execution Time: 0.000066 seconds
Node: bert/encoder/layer_7/attention/output/dense/MatMul, Execution Time: 0.001104 seconds
Node: bert/encoder/layer_7/attention/output/dense/BiasAdd, Execution Time: 0.000485 seconds
Node: bert/encoder/layer_7/attention/output/add, Execution Time: 0.001168 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/moments/mean, Execution Time: 0.000374 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000860 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/moments/SquaredDifference__407, Execution Time: 0.001181 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/moments/variance, Execution Time: 0.000351 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000103 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000075 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/Rsqrt__409, Execution Time: 0.000098 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000099 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000090 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000092 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000788 seconds
Node: bert/encoder/layer_7/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000852 seconds
Node: bert/encoder/layer_7/intermediate/dense/MatMul, Execution Time: 0.002306 seconds
Node: bert/encoder/layer_7/intermediate/dense/BiasAdd, Execution Time: 0.001707 seconds
Node: bert/encoder/layer_7/intermediate/dense/Pow, Execution Time: 0.001998 seconds
Node: bert/encoder/layer_7/intermediate/dense/mul, Execution Time: 0.001988 seconds
Node: bert/encoder/layer_7/intermediate/dense/add, Execution Time: 0.002403 seconds
Node: bert/encoder/layer_7/intermediate/dense/mul_1, Execution Time: 0.001933 seconds
Node: bert/encoder/layer_7/intermediate/dense/Tanh, Execution Time: 0.001862 seconds
Node: bert/encoder/layer_7/intermediate/dense/add_1, Execution Time: 0.001912 seconds
Node: bert/encoder/layer_7/intermediate/dense/mul_2, Execution Time: 0.001880 seconds
Node: bert/encoder/layer_7/intermediate/dense/mul_3, Execution Time: 0.002470 seconds
Node: bert/encoder/layer_7/output/dense/MatMul, Execution Time: 0.002570 seconds
Node: bert/encoder/layer_7/output/dense/BiasAdd, Execution Time: 0.000360 seconds
Node: bert/encoder/layer_7/output/add, Execution Time: 0.000662 seconds
Node: bert/encoder/layer_7/output/LayerNorm/moments/mean, Execution Time: 0.000259 seconds
Node: bert/encoder/layer_7/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000405 seconds
Node: bert/encoder/layer_7/output/LayerNorm/moments/SquaredDifference__411, Execution Time: 0.000582 seconds
Node: bert/encoder/layer_7/output/LayerNorm/moments/variance, Execution Time: 0.000242 seconds
Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/add, Execution Time: 0.000079 seconds
Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000065 seconds
Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/Rsqrt__413, Execution Time: 0.000091 seconds
Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/mul, Execution Time: 0.000083 seconds
Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000076 seconds
Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/sub, Execution Time: 0.000077 seconds
Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000390 seconds
Node: bert/encoder/layer_7/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000442 seconds
Node: bert/encoder/layer_8/attention/self/value/MatMul, Execution Time: 0.000860 seconds
Node: bert/encoder/layer_8/attention/self/value/BiasAdd, Execution Time: 0.000626 seconds
Node: bert/encoder/layer_8/attention/self/Reshape_2, Execution Time: 0.000030 seconds
Node: bert/encoder/layer_8/attention/self/transpose_2, Execution Time: 0.000706 seconds
Node: bert/encoder/layer_8/attention/self/query/MatMul, Execution Time: 0.000869 seconds
Node: bert/encoder/layer_8/attention/self/query/BiasAdd, Execution Time: 0.000605 seconds
Node: bert/encoder/layer_8/attention/self/Reshape, Execution Time: 0.000015 seconds
Node: bert/encoder/layer_8/attention/self/transpose, Execution Time: 0.000648 seconds
Node: bert/encoder/layer_8/attention/self/key/MatMul, Execution Time: 0.000844 seconds
Node: bert/encoder/layer_8/attention/self/key/BiasAdd, Execution Time: 0.000598 seconds
Node: bert/encoder/layer_8/attention/self/Reshape_1, Execution Time: 0.000012 seconds
Node: bert/encoder/layer_8/attention/self/MatMul__418, Execution Time: 0.000650 seconds
Node: bert/encoder/layer_8/attention/self/MatMul, Execution Time: 0.001884 seconds
Node: bert/encoder/layer_8/attention/self/Mul, Execution Time: 0.001476 seconds
Node: bert/encoder/layer_8/attention/self/add, Execution Time: 0.002279 seconds
Node: bert/encoder/layer_8/attention/self/Softmax, Execution Time: 0.002096 seconds
Node: bert/encoder/layer_8/attention/self/MatMul_1, Execution Time: 0.001158 seconds
Node: bert/encoder/layer_8/attention/self/transpose_3, Execution Time: 0.000250 seconds
Node: bert/encoder/layer_8/attention/self/Reshape_3, Execution Time: 0.000042 seconds
Node: bert/encoder/layer_8/attention/output/dense/MatMul, Execution Time: 0.000859 seconds
Node: bert/encoder/layer_8/attention/output/dense/BiasAdd, Execution Time: 0.000352 seconds
Node: bert/encoder/layer_8/attention/output/add, Execution Time: 0.000881 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/moments/mean, Execution Time: 0.000278 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000676 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/moments/SquaredDifference__421, Execution Time: 0.000832 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/moments/variance, Execution Time: 0.000262 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000084 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/Rsqrt__423, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000079 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000076 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000078 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000671 seconds
Node: bert/encoder/layer_8/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000664 seconds
Node: bert/encoder/layer_8/intermediate/dense/MatMul, Execution Time: 0.002161 seconds
Node: bert/encoder/layer_8/intermediate/dense/BiasAdd, Execution Time: 0.001716 seconds
Node: bert/encoder/layer_8/intermediate/dense/Pow, Execution Time: 0.001997 seconds
Node: bert/encoder/layer_8/intermediate/dense/mul, Execution Time: 0.001907 seconds
Node: bert/encoder/layer_8/intermediate/dense/add, Execution Time: 0.002464 seconds
Node: bert/encoder/layer_8/intermediate/dense/mul_1, Execution Time: 0.001903 seconds
Node: bert/encoder/layer_8/intermediate/dense/Tanh, Execution Time: 0.001877 seconds
Node: bert/encoder/layer_8/intermediate/dense/add_1, Execution Time: 0.001901 seconds
Node: bert/encoder/layer_8/intermediate/dense/mul_2, Execution Time: 0.002032 seconds
Node: bert/encoder/layer_8/intermediate/dense/mul_3, Execution Time: 0.002548 seconds
Node: bert/encoder/layer_8/output/dense/MatMul, Execution Time: 0.002471 seconds
Node: bert/encoder/layer_8/output/dense/BiasAdd, Execution Time: 0.000395 seconds
Node: bert/encoder/layer_8/output/add, Execution Time: 0.000648 seconds
Node: bert/encoder/layer_8/output/LayerNorm/moments/mean, Execution Time: 0.000302 seconds
Node: bert/encoder/layer_8/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000407 seconds
Node: bert/encoder/layer_8/output/LayerNorm/moments/SquaredDifference__425, Execution Time: 0.000644 seconds
Node: bert/encoder/layer_8/output/LayerNorm/moments/variance, Execution Time: 0.000256 seconds
Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/add, Execution Time: 0.000088 seconds
Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000070 seconds
Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/Rsqrt__427, Execution Time: 0.000092 seconds
Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/mul, Execution Time: 0.000089 seconds
Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000086 seconds
Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/sub, Execution Time: 0.000076 seconds
Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000392 seconds
Node: bert/encoder/layer_8/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000418 seconds
Node: bert/encoder/layer_9/attention/self/value/MatMul, Execution Time: 0.001007 seconds
Node: bert/encoder/layer_9/attention/self/value/BiasAdd, Execution Time: 0.000638 seconds
Node: bert/encoder/layer_9/attention/self/Reshape_2, Execution Time: 0.000035 seconds
Node: bert/encoder/layer_9/attention/self/transpose_2, Execution Time: 0.000757 seconds
Node: bert/encoder/layer_9/attention/self/query/MatMul, Execution Time: 0.000951 seconds
Node: bert/encoder/layer_9/attention/self/query/BiasAdd, Execution Time: 0.000637 seconds
Node: bert/encoder/layer_9/attention/self/Reshape, Execution Time: 0.000019 seconds
Node: bert/encoder/layer_9/attention/self/transpose, Execution Time: 0.000695 seconds
Node: bert/encoder/layer_9/attention/self/key/MatMul, Execution Time: 0.001056 seconds
Node: bert/encoder/layer_9/attention/self/key/BiasAdd, Execution Time: 0.000635 seconds
Node: bert/encoder/layer_9/attention/self/Reshape_1, Execution Time: 0.000017 seconds
Node: bert/encoder/layer_9/attention/self/MatMul__432, Execution Time: 0.000718 seconds
Node: bert/encoder/layer_9/attention/self/MatMul, Execution Time: 0.001698 seconds
Node: bert/encoder/layer_9/attention/self/Mul, Execution Time: 0.001230 seconds
Node: bert/encoder/layer_9/attention/self/add, Execution Time: 0.001969 seconds
Node: bert/encoder/layer_9/attention/self/Softmax, Execution Time: 0.002053 seconds
Node: bert/encoder/layer_9/attention/self/MatMul_1, Execution Time: 0.001281 seconds
Node: bert/encoder/layer_9/attention/self/transpose_3, Execution Time: 0.000242 seconds
Node: bert/encoder/layer_9/attention/self/Reshape_3, Execution Time: 0.000049 seconds
Node: bert/encoder/layer_9/attention/output/dense/MatMul, Execution Time: 0.001120 seconds
Node: bert/encoder/layer_9/attention/output/dense/BiasAdd, Execution Time: 0.000484 seconds
Node: bert/encoder/layer_9/attention/output/add, Execution Time: 0.001139 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/moments/mean, Execution Time: 0.000366 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000827 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/moments/SquaredDifference__435, Execution Time: 0.001072 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/moments/variance, Execution Time: 0.000333 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000098 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000075 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/Rsqrt__437, Execution Time: 0.000100 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000096 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000086 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000099 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000879 seconds
Node: bert/encoder/layer_9/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000853 seconds
Node: bert/encoder/layer_9/intermediate/dense/MatMul, Execution Time: 0.002726 seconds
Node: bert/encoder/layer_9/intermediate/dense/BiasAdd, Execution Time: 0.001720 seconds
Node: bert/encoder/layer_9/intermediate/dense/Pow, Execution Time: 0.001965 seconds
Node: bert/encoder/layer_9/intermediate/dense/mul, Execution Time: 0.002044 seconds
Node: bert/encoder/layer_9/intermediate/dense/add, Execution Time: 0.002404 seconds
Node: bert/encoder/layer_9/intermediate/dense/mul_1, Execution Time: 0.001914 seconds
Node: bert/encoder/layer_9/intermediate/dense/Tanh, Execution Time: 0.001887 seconds
Node: bert/encoder/layer_9/intermediate/dense/add_1, Execution Time: 0.001947 seconds
Node: bert/encoder/layer_9/intermediate/dense/mul_2, Execution Time: 0.001978 seconds
Node: bert/encoder/layer_9/intermediate/dense/mul_3, Execution Time: 0.002465 seconds
Node: bert/encoder/layer_9/output/dense/MatMul, Execution Time: 0.002364 seconds
Node: bert/encoder/layer_9/output/dense/BiasAdd, Execution Time: 0.000368 seconds
Node: bert/encoder/layer_9/output/add, Execution Time: 0.000608 seconds
Node: bert/encoder/layer_9/output/LayerNorm/moments/mean, Execution Time: 0.000257 seconds
Node: bert/encoder/layer_9/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000440 seconds
Node: bert/encoder/layer_9/output/LayerNorm/moments/SquaredDifference__439, Execution Time: 0.000581 seconds
Node: bert/encoder/layer_9/output/LayerNorm/moments/variance, Execution Time: 0.000251 seconds
Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/add, Execution Time: 0.000089 seconds
Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds
Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/Rsqrt__441, Execution Time: 0.000079 seconds
Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/mul, Execution Time: 0.000078 seconds
Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000075 seconds
Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/sub, Execution Time: 0.000079 seconds
Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000399 seconds
Node: bert/encoder/layer_9/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000434 seconds
Node: bert/encoder/layer_10/attention/self/value/MatMul, Execution Time: 0.000873 seconds
Node: bert/encoder/layer_10/attention/self/value/BiasAdd, Execution Time: 0.000601 seconds
Node: bert/encoder/layer_10/attention/self/Reshape_2, Execution Time: 0.000030 seconds
Node: bert/encoder/layer_10/attention/self/transpose_2, Execution Time: 0.000654 seconds
Node: bert/encoder/layer_10/attention/self/query/MatMul, Execution Time: 0.000870 seconds
Node: bert/encoder/layer_10/attention/self/query/BiasAdd, Execution Time: 0.000624 seconds
Node: bert/encoder/layer_10/attention/self/Reshape, Execution Time: 0.000014 seconds
Node: bert/encoder/layer_10/attention/self/transpose, Execution Time: 0.000667 seconds
Node: bert/encoder/layer_10/attention/self/key/MatMul, Execution Time: 0.000820 seconds
Node: bert/encoder/layer_10/attention/self/key/BiasAdd, Execution Time: 0.000591 seconds
Node: bert/encoder/layer_10/attention/self/Reshape_1, Execution Time: 0.000012 seconds
Node: bert/encoder/layer_10/attention/self/MatMul__446, Execution Time: 0.000669 seconds
Node: bert/encoder/layer_10/attention/self/MatMul, Execution Time: 0.001556 seconds
Node: bert/encoder/layer_10/attention/self/Mul, Execution Time: 0.001174 seconds
Node: bert/encoder/layer_10/attention/self/add, Execution Time: 0.001874 seconds
Node: bert/encoder/layer_10/attention/self/Softmax, Execution Time: 0.001999 seconds
Node: bert/encoder/layer_10/attention/self/MatMul_1, Execution Time: 0.001175 seconds
Node: bert/encoder/layer_10/attention/self/transpose_3, Execution Time: 0.000250 seconds
Node: bert/encoder/layer_10/attention/self/Reshape_3, Execution Time: 0.000041 seconds
Node: bert/encoder/layer_10/attention/output/dense/MatMul, Execution Time: 0.000825 seconds
Node: bert/encoder/layer_10/attention/output/dense/BiasAdd, Execution Time: 0.000351 seconds
Node: bert/encoder/layer_10/attention/output/add, Execution Time: 0.000875 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/moments/mean, Execution Time: 0.000269 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000645 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/moments/SquaredDifference__449, Execution Time: 0.000844 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/moments/variance, Execution Time: 0.000249 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000083 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000060 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/Rsqrt__451, Execution Time: 0.000096 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000085 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000078 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000078 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000643 seconds
Node: bert/encoder/layer_10/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000723 seconds
Node: bert/encoder/layer_10/intermediate/dense/MatMul, Execution Time: 0.002146 seconds
Node: bert/encoder/layer_10/intermediate/dense/BiasAdd, Execution Time: 0.001755 seconds
Node: bert/encoder/layer_10/intermediate/dense/Pow, Execution Time: 0.001994 seconds
Node: bert/encoder/layer_10/intermediate/dense/mul, Execution Time: 0.001992 seconds
Node: bert/encoder/layer_10/intermediate/dense/add, Execution Time: 0.002484 seconds
Node: bert/encoder/layer_10/intermediate/dense/mul_1, Execution Time: 0.001954 seconds
Node: bert/encoder/layer_10/intermediate/dense/Tanh, Execution Time: 0.001888 seconds
Node: bert/encoder/layer_10/intermediate/dense/add_1, Execution Time: 0.001934 seconds
Node: bert/encoder/layer_10/intermediate/dense/mul_2, Execution Time: 0.001957 seconds
Node: bert/encoder/layer_10/intermediate/dense/mul_3, Execution Time: 0.002426 seconds
Node: bert/encoder/layer_10/output/dense/MatMul, Execution Time: 0.002616 seconds
Node: bert/encoder/layer_10/output/dense/BiasAdd, Execution Time: 0.000361 seconds
Node: bert/encoder/layer_10/output/add, Execution Time: 0.000618 seconds
Node: bert/encoder/layer_10/output/LayerNorm/moments/mean, Execution Time: 0.000259 seconds
Node: bert/encoder/layer_10/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000400 seconds
Node: bert/encoder/layer_10/output/LayerNorm/moments/SquaredDifference__453, Execution Time: 0.000586 seconds
Node: bert/encoder/layer_10/output/LayerNorm/moments/variance, Execution Time: 0.000252 seconds
Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/add, Execution Time: 0.000085 seconds
Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000059 seconds
Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/Rsqrt__455, Execution Time: 0.000081 seconds
Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/mul, Execution Time: 0.000078 seconds
Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000077 seconds
Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/sub, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000393 seconds
Node: bert/encoder/layer_10/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000473 seconds
Node: bert/encoder/layer_11/attention/self/value/MatMul, Execution Time: 0.000878 seconds
Node: bert/encoder/layer_11/attention/self/value/BiasAdd, Execution Time: 0.000603 seconds
Node: bert/encoder/layer_11/attention/self/Reshape_2, Execution Time: 0.000030 seconds
Node: bert/encoder/layer_11/attention/self/transpose_2, Execution Time: 0.000656 seconds
Node: bert/encoder/layer_11/attention/self/query/MatMul, Execution Time: 0.000854 seconds
Node: bert/encoder/layer_11/attention/self/query/BiasAdd, Execution Time: 0.000631 seconds
Node: bert/encoder/layer_11/attention/self/Reshape, Execution Time: 0.000017 seconds
Node: bert/encoder/layer_11/attention/self/transpose, Execution Time: 0.000640 seconds
Node: bert/encoder/layer_11/attention/self/key/MatMul, Execution Time: 0.000834 seconds
Node: bert/encoder/layer_11/attention/self/key/BiasAdd, Execution Time: 0.000635 seconds
Node: bert/encoder/layer_11/attention/self/Reshape_1, Execution Time: 0.000012 seconds
Node: bert/encoder/layer_11/attention/self/MatMul__460, Execution Time: 0.000650 seconds
Node: bert/encoder/layer_11/attention/self/MatMul, Execution Time: 0.001605 seconds
Node: bert/encoder/layer_11/attention/self/Mul, Execution Time: 0.001195 seconds
Node: bert/encoder/layer_11/attention/self/add, Execution Time: 0.001884 seconds
Node: bert/encoder/layer_11/attention/self/Softmax, Execution Time: 0.001987 seconds
Node: bert/encoder/layer_11/attention/self/MatMul_1, Execution Time: 0.001057 seconds
Node: bert/encoder/layer_11/attention/self/transpose_3, Execution Time: 0.000238 seconds
Node: bert/encoder/layer_11/attention/self/Reshape_3, Execution Time: 0.000045 seconds
Node: bert/encoder/layer_11/attention/output/dense/MatMul, Execution Time: 0.000828 seconds
Node: bert/encoder/layer_11/attention/output/dense/BiasAdd, Execution Time: 0.000351 seconds
Node: bert/encoder/layer_11/attention/output/add, Execution Time: 0.000849 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/moments/mean, Execution Time: 0.000320 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000652 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/moments/SquaredDifference__463, Execution Time: 0.000820 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/moments/variance, Execution Time: 0.000246 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/add, Execution Time: 0.000086 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000062 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/Rsqrt__465, Execution Time: 0.000083 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul, Execution Time: 0.000085 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000077 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/sub, Execution Time: 0.000078 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000641 seconds
Node: bert/encoder/layer_11/attention/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000716 seconds
Node: bert/encoder/layer_11/intermediate/dense/MatMul, Execution Time: 0.002165 seconds
Node: bert/encoder/layer_11/intermediate/dense/BiasAdd, Execution Time: 0.001761 seconds
Node: bert/encoder/layer_11/intermediate/dense/Pow, Execution Time: 0.001995 seconds
Node: bert/encoder/layer_11/intermediate/dense/mul, Execution Time: 0.002028 seconds
Node: bert/encoder/layer_11/intermediate/dense/add, Execution Time: 0.002431 seconds
Node: bert/encoder/layer_11/intermediate/dense/mul_1, Execution Time: 0.001954 seconds
Node: bert/encoder/layer_11/intermediate/dense/Tanh, Execution Time: 0.002002 seconds
Node: bert/encoder/layer_11/intermediate/dense/add_1, Execution Time: 0.001954 seconds
Node: bert/encoder/layer_11/intermediate/dense/mul_2, Execution Time: 0.001941 seconds
Node: bert/encoder/layer_11/intermediate/dense/mul_3, Execution Time: 0.002417 seconds
Node: bert/encoder/layer_11/output/dense/MatMul, Execution Time: 0.002585 seconds
Node: bert/encoder/layer_11/output/dense/BiasAdd, Execution Time: 0.000364 seconds
Node: bert/encoder/layer_11/output/add, Execution Time: 0.000620 seconds
Node: bert/encoder/layer_11/output/LayerNorm/moments/mean, Execution Time: 0.000267 seconds
Node: bert/encoder/layer_11/output/LayerNorm/moments/SquaredDifference, Execution Time: 0.000401 seconds
Node: bert/encoder/layer_11/output/LayerNorm/moments/SquaredDifference__467, Execution Time: 0.000584 seconds
Node: bert/encoder/layer_11/output/LayerNorm/moments/variance, Execution Time: 0.000235 seconds
Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/add, Execution Time: 0.000081 seconds
Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/Rsqrt, Execution Time: 0.000059 seconds
Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/Rsqrt__469, Execution Time: 0.000079 seconds
Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/mul, Execution Time: 0.000079 seconds
Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/mul_2, Execution Time: 0.000080 seconds
Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/sub, Execution Time: 0.000096 seconds
Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/mul_1, Execution Time: 0.000388 seconds
Node: bert/encoder/layer_11/output/LayerNorm/batchnorm/add_1, Execution Time: 0.000429 seconds
Node: MatMul, Execution Time: 0.048756 seconds
Node: BiasAdd, Execution Time: 0.000149 seconds
Node: Reshape_1, Execution Time: 0.000030 seconds
Node: transpose, Execution Time: 0.000073 seconds
Node: unstack, Execution Time: 0.000066 seconds
Node: unstack__490, Execution Time: 0.000005 seconds
Node: unstack__488, Execution Time: 0.000007 seconds

Total Execution Time: 0.782850 seconds

Total Matmul Fuse Execution Time: 0.000000 seconds
Execution complete.