DEBUG:cmd:2024-05-16 21:23:33:Popen(['git', 'rev-parse', '--show-toplevel'], cwd=/home/luyanfeng/my_code/github/pybind11-OpenKE, stdin=None, shell=False, universal_newlines=False) DEBUG:cmd:2024-05-16 21:23:33:Popen(['git', 'rev-parse', '--show-toplevel'], cwd=/home/luyanfeng/my_code/github/pybind11-OpenKE, stdin=None, shell=False, universal_newlines=False) DEBUG:connectionpool:2024-05-16 21:23:33:Starting new HTTPS connection (1): api.wandb.ai:443 DEBUG:connectionpool:2024-05-16 21:23:34:https://api.wandb.ai:443 "POST /graphql HTTP/1.1" 200 None DEBUG:connectionpool:2024-05-16 21:23:35:https://api.wandb.ai:443 "POST /graphql HTTP/1.1" 200 None wandb: Currently logged in as: 3555028709. Use `wandb login --relogin` to force relogin DEBUG:cmd:2024-05-16 21:23:35:Popen(['git', 'cat-file', '--batch-check'], cwd=/home/luyanfeng/my_code/github/pybind11-OpenKE, stdin=, shell=False, universal_newlines=False) wandb: Tracking run with wandb version 0.16.6 wandb: Run data is saved locally in /home/luyanfeng/my_code/github/pybind11-OpenKE/examples/DistMult/wandb/run-20240516_212335-4aqb2ikm wandb: Run `wandb offline` to turn off syncing. wandb: Syncing run DistMult-WN18RR wandb: ⭐️ View project at https://wandb.ai/3555028709/pybind11-ke wandb: 🚀 View run at https://wandb.ai/3555028709/pybind11-ke/runs/4aqb2ikm INFO:Trainer:2024-05-16 21:23:43:[cuda:1] Initialization completed, start model training. INFO:Trainer:2024-05-16 21:26:24:[cuda:1] Epoch 100 | The model starts evaluation on the validation set. INFO:Trainer:2024-05-16 21:26:27:mr: 5265.586 INFO:Trainer:2024-05-16 21:26:27:mrr: 0.354 INFO:Trainer:2024-05-16 21:26:27:hits@1: 0.3 INFO:Trainer:2024-05-16 21:26:27:hits@3: 0.392 INFO:Trainer:2024-05-16 21:26:27:hits@10: 0.441 INFO:Trainer:2024-05-16 21:26:27:mr_type: 2672.808 INFO:Trainer:2024-05-16 21:26:27:mrr_type: 0.366 INFO:Trainer:2024-05-16 21:26:27:hits@1_type: 0.309 INFO:Trainer:2024-05-16 21:26:27:hits@3_type: 0.405 INFO:Trainer:2024-05-16 21:26:27:hits@10_type: 0.455 INFO:EarlyStopping:2024-05-16 21:26:27:Validation score improved (-inf --> 0.441000). Saving model ... INFO:Trainer:2024-05-16 21:26:27:[cuda:1] Epoch 100 | Training checkpoint saved at ../../checkpoint/distMult-100.pth INFO:Trainer:2024-05-16 21:26:27:[cuda:1] Epoch [ 100/2000] | Batchsize: 4096 | loss: 2.206570 | 1.61467 seconds/epoch INFO:Trainer:2024-05-16 21:28:49:[cuda:1] Epoch 200 | The model starts evaluation on the validation set. INFO:Trainer:2024-05-16 21:28:51:mr: 4623.919 INFO:Trainer:2024-05-16 21:28:51:mrr: 0.336 INFO:Trainer:2024-05-16 21:28:51:hits@1: 0.269 INFO:Trainer:2024-05-16 21:28:51:hits@3: 0.378 INFO:Trainer:2024-05-16 21:28:51:hits@10: 0.449 INFO:Trainer:2024-05-16 21:28:51:mr_type: 2365.995 INFO:Trainer:2024-05-16 21:28:51:mrr_type: 0.35 INFO:Trainer:2024-05-16 21:28:51:hits@1_type: 0.279 INFO:Trainer:2024-05-16 21:28:51:hits@3_type: 0.396 INFO:Trainer:2024-05-16 21:28:51:hits@10_type: 0.466 INFO:EarlyStopping:2024-05-16 21:28:51:Validation score improved (0.441000 --> 0.449000). Saving model ... INFO:Trainer:2024-05-16 21:28:52:[cuda:1] Epoch 200 | Training checkpoint saved at ../../checkpoint/distMult-200.pth INFO:Trainer:2024-05-16 21:28:52:[cuda:1] Epoch [ 200/2000] | Batchsize: 4096 | loss: 2.142169 | 1.52945 seconds/epoch INFO:Trainer:2024-05-16 21:31:08:[cuda:1] Epoch 300 | The model starts evaluation on the validation set. INFO:Trainer:2024-05-16 21:31:10:mr: 4380.868 INFO:Trainer:2024-05-16 21:31:10:mrr: 0.333 INFO:Trainer:2024-05-16 21:31:10:hits@1: 0.261 INFO:Trainer:2024-05-16 21:31:10:hits@3: 0.38 INFO:Trainer:2024-05-16 21:31:10:hits@10: 0.454 INFO:Trainer:2024-05-16 21:31:10:mr_type: 2234.896 INFO:Trainer:2024-05-16 21:31:10:mrr_type: 0.348 INFO:Trainer:2024-05-16 21:31:10:hits@1_type: 0.272 INFO:Trainer:2024-05-16 21:31:10:hits@3_type: 0.397 INFO:Trainer:2024-05-16 21:31:10:hits@10_type: 0.472 INFO:EarlyStopping:2024-05-16 21:31:10:Validation score improved (0.449000 --> 0.454000). Saving model ... INFO:Trainer:2024-05-16 21:31:11:[cuda:1] Epoch 300 | Training checkpoint saved at ../../checkpoint/distMult-300.pth INFO:Trainer:2024-05-16 21:31:11:[cuda:1] Epoch [ 300/2000] | Batchsize: 4096 | loss: 2.113290 | 1.48337 seconds/epoch INFO:Trainer:2024-05-16 21:33:26:[cuda:1] Epoch 400 | The model starts evaluation on the validation set. INFO:Trainer:2024-05-16 21:33:29:mr: 4248.959 INFO:Trainer:2024-05-16 21:33:29:mrr: 0.335 INFO:Trainer:2024-05-16 21:33:29:hits@1: 0.263 INFO:Trainer:2024-05-16 21:33:29:hits@3: 0.382 INFO:Trainer:2024-05-16 21:33:29:hits@10: 0.456 INFO:Trainer:2024-05-16 21:33:29:mr_type: 2169.249 INFO:Trainer:2024-05-16 21:33:29:mrr_type: 0.349 INFO:Trainer:2024-05-16 21:33:29:hits@1_type: 0.273 INFO:Trainer:2024-05-16 21:33:29:hits@3_type: 0.4 INFO:Trainer:2024-05-16 21:33:29:hits@10_type: 0.476 INFO:EarlyStopping:2024-05-16 21:33:29:Validation score improved (0.454000 --> 0.456000). Saving model ... INFO:Trainer:2024-05-16 21:33:29:[cuda:1] Epoch 400 | Training checkpoint saved at ../../checkpoint/distMult-400.pth INFO:Trainer:2024-05-16 21:33:29:[cuda:1] Epoch [ 400/2000] | Batchsize: 4096 | loss: 2.096703 | 1.45826 seconds/epoch INFO:Trainer:2024-05-16 21:35:44:[cuda:1] Epoch 500 | The model starts evaluation on the validation set. INFO:Trainer:2024-05-16 21:35:47:mr: 4166.612 INFO:Trainer:2024-05-16 21:35:47:mrr: 0.335 INFO:Trainer:2024-05-16 21:35:47:hits@1: 0.263 INFO:Trainer:2024-05-16 21:35:47:hits@3: 0.382 INFO:Trainer:2024-05-16 21:35:47:hits@10: 0.459 INFO:Trainer:2024-05-16 21:35:47:mr_type: 2142.348 INFO:Trainer:2024-05-16 21:35:47:mrr_type: 0.349 INFO:Trainer:2024-05-16 21:35:47:hits@1_type: 0.273 INFO:Trainer:2024-05-16 21:35:47:hits@3_type: 0.399 INFO:Trainer:2024-05-16 21:35:47:hits@10_type: 0.48 INFO:EarlyStopping:2024-05-16 21:35:47:Validation score improved (0.456000 --> 0.459000). Saving model ... INFO:Trainer:2024-05-16 21:35:47:[cuda:1] Epoch 500 | Training checkpoint saved at ../../checkpoint/distMult-500.pth INFO:Trainer:2024-05-16 21:35:47:[cuda:1] Epoch [ 500/2000] | Batchsize: 4096 | loss: 2.087295 | 1.44328 seconds/epoch INFO:Trainer:2024-05-16 21:38:01:[cuda:1] Epoch 600 | The model starts evaluation on the validation set. INFO:Trainer:2024-05-16 21:38:04:mr: 4102.903 INFO:Trainer:2024-05-16 21:38:04:mrr: 0.333 INFO:Trainer:2024-05-16 21:38:04:hits@1: 0.258 INFO:Trainer:2024-05-16 21:38:04:hits@3: 0.38 INFO:Trainer:2024-05-16 21:38:04:hits@10: 0.461 INFO:Trainer:2024-05-16 21:38:04:mr_type: 2108.926 INFO:Trainer:2024-05-16 21:38:04:mrr_type: 0.348 INFO:Trainer:2024-05-16 21:38:04:hits@1_type: 0.269 INFO:Trainer:2024-05-16 21:38:04:hits@3_type: 0.396 INFO:Trainer:2024-05-16 21:38:04:hits@10_type: 0.484 INFO:EarlyStopping:2024-05-16 21:38:04:Validation score improved (0.459000 --> 0.461000). Saving model ... INFO:Trainer:2024-05-16 21:38:04:[cuda:1] Epoch 600 | Training checkpoint saved at ../../checkpoint/distMult-600.pth INFO:Trainer:2024-05-16 21:38:04:[cuda:1] Epoch [ 600/2000] | Batchsize: 4096 | loss: 2.080595 | 1.43110 seconds/epoch INFO:Trainer:2024-05-16 21:40:21:[cuda:1] Epoch 700 | The model starts evaluation on the validation set. INFO:Trainer:2024-05-16 21:40:24:mr: 4093.753 INFO:Trainer:2024-05-16 21:40:24:mrr: 0.332 INFO:Trainer:2024-05-16 21:40:24:hits@1: 0.256 INFO:Trainer:2024-05-16 21:40:24:hits@3: 0.383 INFO:Trainer:2024-05-16 21:40:24:hits@10: 0.46 INFO:Trainer:2024-05-16 21:40:24:mr_type: 2098.015 INFO:Trainer:2024-05-16 21:40:24:mrr_type: 0.347 INFO:Trainer:2024-05-16 21:40:24:hits@1_type: 0.267 INFO:Trainer:2024-05-16 21:40:24:hits@3_type: 0.4 INFO:Trainer:2024-05-16 21:40:24:hits@10_type: 0.479 INFO:EarlyStopping:2024-05-16 21:40:24:EarlyStopping counter: 1 / 2 INFO:Trainer:2024-05-16 21:40:24:[cuda:1] Epoch 700 | Training checkpoint saved at ../../checkpoint/distMult-700.pth INFO:Trainer:2024-05-16 21:40:24:[cuda:1] Epoch [ 700/2000] | Batchsize: 4096 | loss: 2.069847 | 1.42638 seconds/epoch INFO:Trainer:2024-05-16 21:42:41:[cuda:1] Epoch 800 | The model starts evaluation on the validation set. INFO:Trainer:2024-05-16 21:42:43:mr: 4087.225 INFO:Trainer:2024-05-16 21:42:43:mrr: 0.333 INFO:Trainer:2024-05-16 21:42:43:hits@1: 0.258 INFO:Trainer:2024-05-16 21:42:43:hits@3: 0.382 INFO:Trainer:2024-05-16 21:42:43:hits@10: 0.46 INFO:Trainer:2024-05-16 21:42:43:mr_type: 2094.564 INFO:Trainer:2024-05-16 21:42:43:mrr_type: 0.347 INFO:Trainer:2024-05-16 21:42:43:hits@1_type: 0.269 INFO:Trainer:2024-05-16 21:42:43:hits@3_type: 0.399 INFO:Trainer:2024-05-16 21:42:43:hits@10_type: 0.48 INFO:EarlyStopping:2024-05-16 21:42:43:EarlyStopping counter: 2 / 2 INFO:Trainer:2024-05-16 21:42:43:[cuda:1] Send an early stopping signal INFO:Trainer:2024-05-16 21:42:43:[cuda:1] The model training is completed, taking a total of 1137.93013 seconds. INFO:Trainer:2024-05-16 21:42:43:[cuda:1] Model saved at ../../checkpoint/distMult.pth. INFO:Trainer:2024-05-16 21:42:43:[cuda:1] The model starts evaluating in the test set. INFO:Trainer:2024-05-16 21:42:46:mr: 3844.078 INFO:Trainer:2024-05-16 21:42:46:mrr: 0.334 INFO:Trainer:2024-05-16 21:42:46:hits@1: 0.259 INFO:Trainer:2024-05-16 21:42:46:hits@3: 0.381 INFO:Trainer:2024-05-16 21:42:46:hits@10: 0.465 INFO:Trainer:2024-05-16 21:42:46:mr_type: 2040.886 INFO:Trainer:2024-05-16 21:42:46:mrr_type: 0.348 INFO:Trainer:2024-05-16 21:42:46:hits@1_type: 0.269 INFO:Trainer:2024-05-16 21:42:46:hits@3_type: 0.399 INFO:Trainer:2024-05-16 21:42:46:hits@10_type: 0.483 wandb: - 0.023 MB of 0.038 MB uploaded wandb: \ 0.024 MB of 0.038 MB uploaded wandb: | 0.034 MB of 0.038 MB uploaded wandb: / 0.038 MB of 0.038 MB uploaded wandb: - 0.038 MB of 0.038 MB uploaded wandb: \ 0.038 MB of 0.038 MB uploaded wandb: wandb: wandb: Run history: wandb: duration ▁ wandb: test/hits@1 ▁ wandb: test/hits@10 ▁ wandb: test/hits@10_type ▁ wandb: test/hits@1_type ▁ wandb: test/hits@3 ▁ wandb: test/hits@3_type ▁ wandb: test/mr ▁ wandb: test/mr_type ▁ wandb: test/mrr ▁ wandb: test/mrr_type ▁ wandb: train/epoch ▁▂▃▅▆▇█ wandb: train/train_loss █▅▃▂▂▂▁ wandb: val/epoch ▁▂▃▄▅▆▇█ wandb: val/hits@1 █▃▂▂▂▁▁▁ wandb: val/hits@10 ▁▄▆▆▇███ wandb: val/hits@10_type ▁▄▅▆▇█▇▇ wandb: val/hits@1_type █▃▂▂▂▁▁▁ wandb: val/hits@3 █▁▂▃▃▂▃▃ wandb: val/hits@3_type █▁▂▄▃▁▄▃ wandb: val/mr █▄▃▂▁▁▁▁ wandb: val/mr_type █▄▃▂▂▁▁▁ wandb: val/mrr █▂▁▂▂▁▁▁ wandb: val/mrr_type █▂▁▂▂▁▁▁ wandb: wandb: Run summary: wandb: duration 1137.93013 wandb: test/hits@1 0.259 wandb: test/hits@10 0.465 wandb: test/hits@10_type 0.483 wandb: test/hits@1_type 0.269 wandb: test/hits@3 0.381 wandb: test/hits@3_type 0.399 wandb: test/mr 3844.078 wandb: test/mr_type 2040.886 wandb: test/mrr 0.334 wandb: test/mrr_type 0.348 wandb: train/epoch 700 wandb: train/train_loss 2.06985 wandb: val/epoch 799 wandb: val/hits@1 0.258 wandb: val/hits@10 0.46 wandb: val/hits@10_type 0.48 wandb: val/hits@1_type 0.269 wandb: val/hits@3 0.382 wandb: val/hits@3_type 0.399 wandb: val/mr 4087.225 wandb: val/mr_type 2094.564 wandb: val/mrr 0.333 wandb: val/mrr_type 0.347 wandb: wandb: 🚀 View run DistMult-WN18RR at: https://wandb.ai/3555028709/pybind11-ke/runs/4aqb2ikm wandb: ⭐️ View project at: https://wandb.ai/3555028709/pybind11-ke wandb: Synced 5 W&B file(s), 0 media file(s), 2 artifact file(s) and 0 other file(s) wandb: Find logs at: ./wandb/run-20240516_212335-4aqb2ikm/logs DEBUG:connectionpool:2024-05-16 21:43:00:Starting new HTTPS connection (1): o151352.ingest.sentry.io:443 DEBUG:connectionpool:2024-05-16 21:43:01:https://o151352.ingest.sentry.io:443 "POST /api/4504800232407040/envelope/ HTTP/1.1" 200 0