Zchen
3c993a6268
Increase safety margin to 30% in dataset shape analysis for improved padding accuracy
2025-10-22 01:47:08 +08:00
Zchen
6fb5907c72
Refactor create_input_fn to support static shape handling for XLA compatibility
2025-10-22 01:29:31 +08:00
Zchen
c03441d8f3
Refactor dynamic padding shapes and update device placement configuration for TPU training
2025-10-22 01:03:14 +08:00
Zchen
57f07434ac
f
2025-10-22 00:54:20 +08:00
Zchen
52a9b17375
f
2025-10-22 00:38:55 +08:00
Zchen
e715d9ac79
Enhance error handling and deprecate batch generation methods in BrainToTextDatasetTF
...
- Improved error logging when loading trial data fails, ensuring correct feature dimensions in dummy data.
- Marked _create_batch_generator and create_dataset methods as deprecated, recommending create_input_fn for better performance.
- Adjusted maximum parallel workers in analyze_dataset_shapes based on CPU cores.
2025-10-22 00:28:10 +08:00
Zchen
a031972ba6
f
2025-10-21 01:07:57 +08:00
Zchen
ab12d0b7ee
f
2025-10-21 00:31:59 +08:00
Zchen
e7c9b95b00
f
2025-10-21 00:19:05 +08:00
Zchen
5a0079641a
f
2025-10-20 23:34:44 +08:00
Zchen
e399cf262a
ff
2025-10-20 13:37:11 +08:00
Zchen
7358ff3d79
Enable soft device placement for CTC operations and update related comments
2025-10-20 11:22:13 +08:00
Zchen
f8fb4d7133
Remove setup script, TPU memory monitor, and training model script
...
- Deleted `setup_tensorflow_tpu.sh` which was responsible for setting up the TensorFlow environment on TPU v5e-8.
- Removed `tpu_memory_monitor.py`, a tool for monitoring TPU memory usage during training.
- Eliminated `train_model.py`, the script for training the Brain-to-Text RNN model.
2025-10-20 11:05:03 +08:00
Zchen
7c272b7c5b
Remove test scripts for data loading and TensorFlow implementation
2025-10-20 01:37:22 +08:00
Zchen
0a0e07a193
Remove custom CTC loss implementation for TPU from the TripleGRUDecoder class
2025-10-20 01:16:50 +08:00
Zchen
06ddbc6ac2
Refactor input function to implement batch-first approach with dynamic padding and apply data augmentation post-batching for TPU compatibility
2025-10-20 00:58:29 +08:00
Zchen
fabf70cfa9
Enhance dataset shape analysis by implementing parallel processing and improving sampling logic
2025-10-20 00:35:17 +08:00
Zchen
e1669b5a4c
Increase batch size from 256 to 512 for training in rnn_args.yaml
2025-10-20 00:21:33 +08:00
Zchen
6e02894a8a
f
2025-10-20 00:13:39 +08:00
Zchen
4db3625dc5
f
2025-10-19 23:55:56 +08:00
Zchen
fed5fd8251
f
2025-10-19 22:25:21 +08:00
Zchen
4b373ab317
ff
2025-10-19 20:16:23 +08:00
Zchen
40d0fc50de
f
2025-10-19 13:18:20 +08:00
Zchen
4328114ed6
Add dataset shape analysis function and integrate into input function for TPU optimization
2025-10-19 11:04:36 +08:00
Zchen
cfd9653da9
Enhance dataset caching logic for training and validation sets with improved messaging
2025-10-19 10:31:31 +08:00
Zchen
558be0ad98
Refactor individual dataset creation for improved I/O efficiency and add logging for error handling
2025-10-19 10:31:18 +08:00
Zchen
d83f990beb
f
2025-10-17 12:20:17 +08:00
Zchen
eb058fe9d3
ff
2025-10-17 11:57:10 +08:00
Zchen
57360bec8a
Remove CPU optimization call and add logging for TPU strategy and data pipeline performance
2025-10-17 11:45:20 +08:00
Zchen
eb4e3fc69f
fff
2025-10-17 11:38:57 +08:00
Zchen
6c7abfcca8
f
2025-10-17 10:53:58 +08:00
Zchen
7ede7b5f12
f
2025-10-17 02:09:14 +08:00
Zchen
ca8c615505
f
2025-10-17 02:01:48 +08:00
Zchen
49700456b8
f
2025-10-17 01:58:28 +08:00
Zchen
8ee09b6b5e
f
2025-10-17 01:54:32 +08:00
Zchen
a5a3179ca6
f
2025-10-17 01:49:03 +08:00
Zchen
59fb73ee9f
f
2025-10-17 01:36:08 +08:00
Zchen
0a72143513
legacy adam
2025-10-17 01:26:02 +08:00
Zchen
7df78244e6
adamw to adam
2025-10-17 01:07:01 +08:00
Zchen
a96e272f7b
fix twice gradient cut
2025-10-17 00:51:53 +08:00
Zchen
7a43ebfb71
refactor: streamline model building and ensure dtype consistency in L2 loss calculation
2025-10-16 23:06:09 +08:00
Zchen
9453b70fad
remove quick test script for TensorFlow implementation fixes
2025-10-16 23:05:53 +08:00
Zchen
7efa33d730
f
2025-10-16 22:42:33 +08:00
Zchen
982d2dc256
f
2025-10-16 22:20:08 +08:00
Zchen
bd61136f93
f
2025-10-16 22:02:11 +08:00
Zchen
6f94ad5fae
f
2025-10-16 21:51:43 +08:00
Zchen
eefff1ce5e
fix
2025-10-16 21:40:43 +08:00
Zchen
426b72ef25
fix
2025-10-16 21:26:00 +08:00
Zchen
dde6378481
fixed
2025-10-16 21:13:42 +08:00
Zchen
a0b59c6987
fix
2025-10-16 21:06:01 +08:00
