-------------------------------------------------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  
                                                   Name    Self CPU %      Self CPU   CPU total %     CPU total  CPU time avg     Self CUDA   Self CUDA %    CUDA total  CUDA time avg       CPU Mem  Self CPU Mem      CUDA Mem  Self CUDA Mem    # of Calls  
-------------------------------------------------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  
                                            aten::addmm         5.29%     221.800ms        10.79%     452.443ms      46.409us        1.310s        27.48%        1.478s     151.566us           0 B           0 B     245.33 GB     245.33 GB          9749  
                                    Command Buffer Full        61.13%        2.563s        61.13%        2.563s     122.994us        1.294s        27.13%        1.294s      62.070us           0 B           0 B           0 B           0 B         20840  
                         aten::_flash_attention_forward         0.18%       7.737ms         0.93%      39.203ms      51.582us        1.010s        21.18%        1.268s       1.668ms           0 B           0 B      31.52 GB           0 B           760  
void pytorch_flash::flash_fwd_kernel<Flash_fwd_kerne...         0.00%       0.000us         0.00%       0.000us       0.000us        1.010s        21.18%        1.010s       1.328ms           0 B           0 B           0 B           0 B           760  
         nvjet_tst_128x256_64x4_1x2_h_bz_coopA_bias_TNN         0.00%       0.000us         0.00%       0.000us       0.000us     724.633ms        15.20%     724.633ms     238.366us           0 B           0 B           0 B           0 B          3040  
                                            aten::copy_         1.25%      52.221ms        14.43%     605.100ms      57.590us     514.843ms        10.80%     572.747ms      54.511us           0 B         -80 B           0 B           0 B         10507  
                                              aten::mul         2.03%      85.066ms        24.79%        1.039s      81.770us     493.232ms        10.35%     610.108ms      47.995us           0 B           0 B     308.22 GB     308.22 GB         12712  
         nvjet_tst_192x192_64x4_1x1_h_bz_coopB_bias_TNN         0.00%       0.000us         0.00%       0.000us       0.000us     394.555ms         8.28%     394.555ms     519.151us           0 B           0 B           0 B           0 B           760  
void at::native::elementwise_kernel<128, 4, at::nati...         0.00%       0.000us         0.00%       0.000us       0.000us     364.792ms         7.65%     364.792ms      59.432us           0 B           0 B           0 B           0 B          6138  
                                              aten::add         1.93%      80.725ms        12.93%     542.028ms      42.649us     343.008ms         7.19%     451.088ms      35.494us         224 B         224 B     128.00 GB     128.00 GB         12709  
void at::native::elementwise_kernel<128, 4, at::nati...         0.00%       0.000us         0.00%       0.000us       0.000us     310.842ms         6.52%     310.842ms      76.979us           0 B           0 B           0 B           0 B          4038  
void at::native::elementwise_kernel<128, 4, at::nati...         0.00%       0.000us         0.00%       0.000us       0.000us     304.582ms         6.39%     304.582ms      33.186us           0 B           0 B           0 B           0 B          9178  
                                              aten::cat         0.36%      15.180ms         2.64%     110.592ms      46.881us     284.981ms         5.98%     548.478ms     232.504us         416 B         416 B      92.71 GB      92.71 GB          2359  
void at::native::(anonymous namespace)::CatArrayBatc...         0.00%       0.000us         0.00%       0.000us       0.000us     284.816ms         5.97%     284.816ms     124.920us           0 B           0 B           0 B           0 B          2280  
void at::native::elementwise_kernel<128, 4, at::nati...         0.00%       0.000us         0.00%       0.000us       0.000us     187.388ms         3.93%     187.388ms      59.716us           0 B           0 B           0 B           0 B          3138  
-------------------------------------------------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  ------------  
Self CPU time total: 4.193s
Self CUDA time total: 4.768s
