Commit Graph
560 Commits
Author SHA1 Message Date
Glenn Jocher 22ee6fb7c1 Update is_writeable() for 2 methods (#4744)
* Writeable test

* Fix

* Cleanup
2021-09-10 17:52:33 +02:00
Glenn Jocher a144536f88 Fix is_writeable() for 3 OS support (#4743)
* Fix `is_writeable()` for 3 OS support

* Update general.py
2021-09-10 17:06:22 +02:00
Josh Veitch-MichaelisandGlenn Jocher 19e28e3bfe Add callbacks to train function in W&B sweep (#4742)
* add callbacks to train function in wandb sweep

Fix following https://github.com/ultralytics/yolov5/pull/4688 which modified the function signature to `train`

* Cleanup

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-09-10 16:28:52 +02:00
Glenn Jocher 4a025ae97f Fix user_config_dir() for GCP/AWS functions (#4726)
* Fix `user_config_dir()` for GCP/AWS functions

Compatability fix for GCP functions and AWS lambda for user config directory in https://github.com/ultralytics/yolov5/pull/4628

* Windows skip check
2021-09-09 17:57:46 +02:00
JD Costa deb434aefa Make CONFIG_DIR configurable per environment variable (#4727) 2021-09-09 15:29:18 +02:00
Glenn Jocher 8e94bf62d9 Add user_config_dir('Ultralytics') (#4715)
* Add `user_config_dir`

* Linux to .config
2021-09-08 18:13:59 +02:00
Glenn Jocher 25a7e1dae5 Update check_yaml() comment (#4713)
* Update `check_yaml()` comment

* Cleanup
2021-09-08 16:01:03 +02:00
Glenn Jocher f984cce52a Fix check_suffix() (#4712)
Fix a bug when `file=''`
2021-09-08 15:06:31 +02:00
Glenn Jocher a2b3c71636 Add suffix checks (#4711)
* Add suffix checks

* Cleanup

* Cleanup2

* Cleanup3
2021-09-08 14:36:12 +02:00
Zhiqiang Wang 8e5f9ddbdb Remove redundant ComputeLoss code (#4701) 2021-09-08 12:48:33 +02:00
Kalen MichaelandGlenn Jocher 2317f86ca4 Optimised Callback Class to Reduce Code and Fix Errors (#4688)
* added callbacks

* added back callback to main

* added save_dir to callback output

* reduced code count

* updated callbacks

* added default callback class to main, added missing parameters to on_model_save

* Glenn updates

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-09-07 18:32:15 +02:00
Glenn Jocher 548745181a EarlyStopper updates (#4679) 2021-09-05 19:09:53 +02:00
Glenn Jocher 234e8ae6fa Fix for plot_evolve() string argument (#4639) 2021-09-01 15:00:13 +02:00
Glenn Jocher de534e9221 Fix 3 for Arial.ttf redownloads with hub inference (#4629)
Fix 3 for Arial.ttf redownloads with hub inference, follow-on to #4628.
2021-08-31 20:54:10 +02:00
Glenn Jocher a4e8f78c5e Fix 2 for Arial.ttf redownloads with hub inference (#4628) 2021-08-31 20:05:17 +02:00
Glenn Jocher ba0f80874f Fix for Arial.ttf redownloads with hub inference (#4627) 2021-08-31 15:01:41 +02:00
Glenn Jocher 50a9828679 DDP torch.jit.trace() --sync-bn fix (#4615)
* Remove assert

* debug0

* trace=not opt.sync

* sync to sync_bn fix

* Cleanup
2021-08-30 18:35:07 +02:00
Glenn Jocher bb5ebc290e Close matplotlib plots after opening (#4612)
* Close plots

* Replace fig.close() for plt.close()
2021-08-30 17:22:21 +02:00
Ayush Chaurasia 11e7c7b48d W&B: Don't log models in evolve operation (#4611) 2021-08-30 17:07:20 +02:00
Glenn Jocher 3a72d4a7e3 Update check_git_status() warning (#4610) 2021-08-30 17:05:45 +02:00
Glenn Jocher e7d1842a98 Auto-download Arial.ttf on init (#4606)
* Auto-download Arial.ttf on init

* Fix ROOT
2021-08-30 14:58:22 +02:00
Glenn Jocher dc13820c9d TensorBoard on_train_end() speed improvements (#4605) 2021-08-30 13:01:41 +02:00
Glenn Jocher b894e69dfc Update mosaic plots font size (#4596) 2021-08-29 18:05:49 +02:00
Glenn Jocher dbbc6b5c48 Re-order plots.py to class-first (#4595) 2021-08-29 17:44:51 +02:00
Glenn Jocher e5e5ebc799 Auto-UTF handling (#4594) 2021-08-29 17:15:18 +02:00
Glenn Jocher de44376d1b Create Annotator() class (#4591)
* Add Annotator() class

* Download Arial

* 2x for loop

* Cleanup

* tuple 2 list

* max_size=1920

* bold logging results to

* tolist()

* im = annotator.im

* PIL save in detect.py

* Smart asarray in detect.py

* revert to cv2.imwrite

* Cleanup

* Return result asarray

* Add `Profile()` profiler

* CamelCase Timeout

* Resize after mosaic

* pillow>=8.0.0

* daemon imwrite

* Add cv2 support

* Remove plot_wh_methods and plot_one_box

* pil=False for hubconf.py annotations

* im.shape bug fix

* colorstr common.py

* join daemons

* Update t.daemon

* Removed daemon saving
2021-08-29 16:46:13 +02:00
Takumi Karasawa 7b35971ba5 Fix bug in plot_one_box when label is None (#4588) 2021-08-29 15:08:27 +02:00
Glenn Jocher bbfafeabdb Add Profile() profiler (#4587)
* Add `Profile()` profiler

* CamelCase Timeout
2021-08-29 13:49:04 +02:00
Glenn Jocher 93cc015748 Add EarlyStopping feature (#4576)
* Add EarlyStopping feature

* Add comment

* Cleanup

* Cleanup2

* debug

* debug2

* debug3

* debug3

* debug4

* debug5

* debug6

* debug7

* debug8

* debug9

* debug10

* debug11

* debug12

* Cleanup

* Add TODO for known DDP issue
2021-08-28 19:03:52 +02:00
Glenn Jocher 8ac96b7975 Fix for AP calculation limits 0.0 - 1.0 (#4563)
This PR brings alignment in AP computation practices with Detectron2 and MMDetection. 

Problem first noted by @yusiyoh in https://github.com/ultralytics/yolov5/issues/4546
2021-08-27 13:23:29 +02:00
Glenn Jocher e899d6e8fb Fix for corrupt JPEGs auto-fix PR (#4560)
Auto-fix corrupt JPEGs PR introduced a bug whereby the f.seek() operation read all of the bytes in the image, resulting in the PIL image having nothing to read upon the .save() operation. 

Fix was to re-open the image using PIL before saving.
2021-08-27 13:01:21 +02:00
Glenn Jocher 11f85e7e71 Auto-fix corrupt JPEGs (#4548)
* Autofix corrupt JPEGs

This PR automatically re-saves corrupt JPEGs and trains with the resaved images. WARNING: this will overwrite the existing corrupt JPEGs in a dataset and replace them with correct JPEGs, though the filesize may increase and the image contents may not be exactly the same due to lossy JPEG compression schemes. Results may vary by JPEG decoder and hardware.

Current behavior is to exclude corrupt JPEGs from training with a warning to the user, but many users have been complaining about large parts of their dataset being excluded from training.

* Clarify re-save reason
2021-08-26 15:51:04 +02:00
Glenn Jocher 7b1643b5b5 Add install=True argument to check_requirements (#4512)
* Add `install=True` argument to `check_requirements`

* Update general.py
2021-08-23 14:38:30 +02:00
Ayush ChaurasiaandGlenn Jocher 7316b78e36 W&B: Refactor the wandb_utils.py file (#4496)
* Improve docstrings and run names

* default wandb login prompt with timeout

* return key

* Update api_key check logic

* Properly support zipped dataset feature

* update docstring

* Revert tuorial change

* extend changes to log_dataset

* add run name

* bug fix

* bug fix

* Update comment

* fix import check

* remove unused import

* Hardcore .yaml file extension

* reduce code

* Reformat using pycharm

* Remove redundant try catch

* More refactoring and bug fixes

* retry

* Reformat using pycharm

* respect LOGGERS include list

* Fix

* fix

* refactor constructor

* refactor

* refactor

* refactor

* PyCharm reformat

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-08-23 13:10:07 +02:00
1d65e8194d Fix missing labels after albumentations (#4455)
* fix missing labels after augmentation

* Update datasets.py

Cleanup

Co-authored-by: Huu Quan <huuquan@HuuQuans-MacBook.local>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-08-18 12:07:09 +02:00
Jiacong FangandGlenn Jocher 808bcad3bb Add TensorFlow and TFLite export (#1127)
* Add models/tf.py for TensorFlow and TFLite export

* Set auto=False for int8 calibration

* Update requirements.txt for TensorFlow and TFLite export

* Read anchors directly from PyTorch weights

* Add --tf-nms to append NMS in TensorFlow SavedModel and GraphDef export

* Remove check_anchor_order, check_file, set_logging from import

* Reformat code and optimize imports

* Autodownload model and check cfg

* update --source path, img-size to 320, single output

* Adjust representative_dataset

* Put representative dataset in tfl_int8 block

* detect.py TF inference

* weights to string

* weights to string

* cleanup tf.py

* Add --dynamic-batch-size

* Add xywh normalization to reduce calibration error

* Update requirements.txt

TensorFlow 2.3.1 -> 2.4.0 to avoid int8 quantization error

* Fix imports

Move C3 from models.experimental to models.common

* Add models/tf.py for TensorFlow and TFLite export

* Set auto=False for int8 calibration

* Update requirements.txt for TensorFlow and TFLite export

* Read anchors directly from PyTorch weights

* Add --tf-nms to append NMS in TensorFlow SavedModel and GraphDef export

* Remove check_anchor_order, check_file, set_logging from import

* Reformat code and optimize imports

* Autodownload model and check cfg

* update --source path, img-size to 320, single output

* Adjust representative_dataset

* detect.py TF inference

* Put representative dataset in tfl_int8 block

* weights to string

* weights to string

* cleanup tf.py

* Add --dynamic-batch-size

* Add xywh normalization to reduce calibration error

* Update requirements.txt

TensorFlow 2.3.1 -> 2.4.0 to avoid int8 quantization error

* Fix imports

Move C3 from models.experimental to models.common

* implement C3() and SiLU()

* Fix reshape dim to support dynamic batching

* Add epsilon argument in tf_BN, which is different between TF and PT

* Set stride to None if not using PyTorch, and do not warmup without PyTorch

* Add list support in check_img_size()

* Add list input support in detect.py

* sys.path.append('./') to run from yolov5/

* Add int8 quantization support for TensorFlow 2.5

* Add get_coco128.sh

* Remove --no-tfl-detect in models/tf.py (Use tf-android-tfl-detect branch for EdgeTPU)

* Update requirements.txt

* Replace torch.load() with attempt_load()

* Update requirements.txt

* Add --tf-raw-resize to set half_pixel_centers=False

* Add --agnostic-nms for TF class-agnostic NMS

* Cleanup after merge

* Cleanup2 after merge

* Cleanup3 after merge

* Add tf.py docstring with credit and usage

* pb saved_model and tflite use only one model in detect.py

* Add use cases in docstring of tf.py

* Remove redundant `stride` definition

* Remove keras direct import

* Fix `check_requirements(('tensorflow>=2.4.1',))`

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-08-17 13:18:16 +02:00
Glenn Jocher f3e3f7603f TFLite prep (#4436) 2021-08-16 17:25:06 +02:00
Glenn Jocher 19d03a955c Remove DDP process group timeout (#4422) 2021-08-15 18:32:41 +02:00
Glenn Jocher 24bea5e4b7 Standardize headers and docstrings (#4417)
* Implement new headers

* Reformat 1

* Reformat 2

* Reformat 3 - math

* Reformat 4 - yaml
2021-08-14 21:17:51 +02:00
Glenn Jocher 2da4e7acf7 Merge PIL and OpenCV in plot_one_box(use_pil=False) (#4416)
* Merge PIL and OpenCV box plotting functions

* Add ASCII check to plot_one_box

* Cleanup

* Cleanup2
2021-08-14 17:44:15 +02:00
Glenn Jocher 63e09fdc48 Remove encoding='ascii' (#4413)
* Remove `encoding='ascii'`

* Reinstate `encoding='ascii'` in emojis()
2021-08-14 13:47:20 +02:00
Glenn Jocher e086347377 Update plots.py (#4407) 2021-08-14 12:50:27 +02:00
Glenn Jocher 3e7c59ad3b Fix rename utils.google_utils to utils.downloads (#4393) 2021-08-11 23:40:47 +02:00
Ayush ChaurasiaandGlenn Jocher a4e4553040 W&B: Fix for 4360 (#4388)
* Improve docstrings and run names

* default wandb login prompt with timeout

* return key

* Update api_key check logic

* Properly support zipped dataset feature

* update docstring

* Revert tuorial change

* extend changes to log_dataset

* add run name

* bug fix

* bug fix

* Update comment

* fix import check

* remove unused import

* Hardcore .yaml file extension

* reduce code

* Reformat using pycharm

* Remove redundant try catch

* More refactoring and bug fixes

* retry

* Reformat using pycharm

* respect LOGGERS include list

* Fix

* fix

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-08-11 21:56:25 +02:00
Ayush ChaurasiaandGlenn Jocher 75d323dd8b W&B: Add advanced features tutorial (#4384)
* Improve docstrings and run names

* default wandb login prompt with timeout

* return key

* Update api_key check logic

* Properly support zipped dataset feature

* update docstring

* Revert tuorial change

* extend changes to log_dataset

* add run name

* bug fix

* bug fix

* Update comment

* fix import check

* remove unused import

* Hardcore .yaml file extension

* reduce code

* Reformat using pycharm

* Remove redundant try catch

* More refactoring and bug fixes

* retry

* Reformat using pycharm

* respect LOGGERS include list

* Initial readme update

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-08-11 18:35:43 +02:00
Sudhanshu Singh f409d8e54f Update README.md (#4309)
remove unnecessary "`"
2021-08-04 22:11:48 +02:00
Glenn Jocher e78aeac973 Evolve in CSV format (#4307)
* Update evolution to CSV format

* Update

* Update

* Update

* Update

* Update

* reset args

* reset args

* reset args

* plot_results() fix

* Cleanup

* Cleanup2
2021-08-04 17:13:38 +02:00
Glenn Jocher 4103ce9ad0 Simplify callbacks (#4289) 2021-08-04 13:17:35 +02:00
imyhxyandfkwong 771ac6c53d Fixed logging level in distributed mode (#4284)
Co-authored-by: fkwong <huangfuqiang@transai.cn>
2021-08-03 13:11:42 +02:00
junji hashimotoandGlenn Jocher 2d99063201 Feature python train.py --cache disk (#4049)
* Add cache-on-disk and cache-directory to cache images on disk

* Fix load_image with cache_on_disk

* Add no_cache flag for load_image

* Revert the parts('logging' and a new line) that do not need to be modified

* Add the assertion for shapes of cached images

* Add a suffix string for cached images

* Fix boundary-error of letterbox for load_mosaic

* Add prefix as cache-key of cache-on-disk

* Update cache-function on disk

* Add psutil in requirements.txt

* Update train.py

* Cleanup1

* Cleanup2

* Skip existing npy

* Include re-space

* Export return character fix

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-08-02 18:47:24 +02:00