Add EdgeTPU support (#3630)

* Add models/tf.py for TensorFlow and TFLite export

* Set auto=False for int8 calibration

* Update requirements.txt for TensorFlow and TFLite export

* Read anchors directly from PyTorch weights

* Add --tf-nms to append NMS in TensorFlow SavedModel and GraphDef export

* Remove check_anchor_order, check_file, set_logging from import

* Reformat code and optimize imports

* Autodownload model and check cfg

* update --source path, img-size to 320, single output

* Adjust representative_dataset

* Put representative dataset in tfl_int8 block

* detect.py TF inference

* weights to string

* weights to string

* cleanup tf.py

* Add --dynamic-batch-size

* Add xywh normalization to reduce calibration error

* Update requirements.txt

TensorFlow 2.3.1 -> 2.4.0 to avoid int8 quantization error

* Fix imports

Move C3 from models.experimental to models.common

* Add models/tf.py for TensorFlow and TFLite export

* Set auto=False for int8 calibration

* Update requirements.txt for TensorFlow and TFLite export

* Read anchors directly from PyTorch weights

* Add --tf-nms to append NMS in TensorFlow SavedModel and GraphDef export

* Remove check_anchor_order, check_file, set_logging from import

* Reformat code and optimize imports

* Autodownload model and check cfg

* update --source path, img-size to 320, single output

* Adjust representative_dataset

* detect.py TF inference

* Put representative dataset in tfl_int8 block

* weights to string

* weights to string

* cleanup tf.py

* Add --dynamic-batch-size

* Add xywh normalization to reduce calibration error

* Update requirements.txt

TensorFlow 2.3.1 -> 2.4.0 to avoid int8 quantization error

* Fix imports

Move C3 from models.experimental to models.common

* implement C3() and SiLU()

* Add TensorFlow and TFLite Detection

* Add --tfl-detect for TFLite Detection

* Add int8 quantized TFLite inference in detect.py

* Add --edgetpu for Edge TPU detection

* Fix --img-size to add rectangle TensorFlow and TFLite input

* Add --no-tf-nms to detect objects using models combined with TensorFlow NMS

* Fix --img-size list type input

* Update README.md

* Add Android project for TFLite inference

* Upgrade TensorFlow v2.3.1 -> v2.4.0

* Disable normalization of xywh

* Rewrite names init in detect.py

* Change input resolution 640 -> 320 on Android

* Disable NNAPI

* Update README.me --img 640 -> 320

* Update README.me for Edge TPU

* Update README.md

* Fix reshape dim to support dynamic batching

* Fix reshape dim to support dynamic batching

* Add epsilon argument in tf_BN, which is different between TF and PT

* Set stride to None if not using PyTorch, and do not warmup without PyTorch

* Add list support in check_img_size()

* Add list input support in detect.py

* sys.path.append('./') to run from yolov5/

* Add int8 quantization support for TensorFlow 2.5

* Add get_coco128.sh

* Remove --no-tfl-detect in models/tf.py (Use tf-android-tfl-detect branch for EdgeTPU)

* Update requirements.txt

* Replace torch.load() with attempt_load()

* Update requirements.txt

* Add --tf-raw-resize to set half_pixel_centers=False

* Remove android directory

* Update README.md

* Update README.md

* Add multiple OS support for EdgeTPU detection

* Fix export and detect

* Export 3 YOLO heads with Edge TPU models

* Remove xywh denormalization with Edge TPU models in detect.py

* Fix saved_model and pb detect error

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix pre-commit.ci failure

* Add edgetpu in export.py docstring

* Fix Edge TPU model detection exported by TF 2.7

* Add class names for TF/TFLite in DetectMultibackend

* Fix assignment with nl in TFLite Detection

* Add check when getting Edge TPU compiler version

* Add UTF-8 encoding in opening --data file for Windows

* Remove redundant TensorFlow import

* Add Edge TPU in export.py's docstring

* Add the detect layer in Edge TPU model conversion

* Default `dnn=False`

* Cleanup data.yaml loading

* Update detect.py

* Update val.py

* Comments and generalize data.yaml names

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: unknown <fangjiacong@ut.cn>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

models/common.py

@@ -17,6 +17,7 @@ import pandas as pd
 import requests
 import torch
 import torch.nn as nn
+import yaml
 from PIL import Image
 from torch.cuda import amp
 
@@ -276,7 +277,7 @@ class Concat(nn.Module):
 
 class DetectMultiBackend(nn.Module):
     # YOLOv5 MultiBackend class for python inference on various backends
-    def __init__(self, weights='yolov5s.pt', device=None, dnn=False):
+    def __init__(self, weights='yolov5s.pt', device=None, dnn=False, data=None):
         # Usage:
         #   PyTorch:      weights = *.pt
         #   TorchScript:            *.torchscript
@@ -284,6 +285,7 @@ class DetectMultiBackend(nn.Module):
         #   TensorFlow:             *_saved_model
         #   TensorFlow:             *.pb
         #   TensorFlow Lite:        *.tflite
+        #   TensorFlow Edge TPU:    *_edgetpu.tflite
         #   ONNX Runtime:           *.onnx
         #   OpenCV DNN:             *.onnx with dnn=True
         #   TensorRT:               *.engine
@@ -297,6 +299,9 @@ class DetectMultiBackend(nn.Module):
         pt, jit, onnx, engine, tflite, pb, saved_model, coreml = (suffix == x for x in suffixes)  # backend booleans
         stride, names = 64, [f'class{i}' for i in range(1000)]  # assign defaults
         w = attempt_download(w)  # download if not local
+        if data:  # data.yaml path (optional)
+            with open(data, errors='ignore') as f:
+                names = yaml.safe_load(f)['names']  # class names
 
         if jit:  # TorchScript
             LOGGER.info(f'Loading {w} for TorchScript inference...')
@@ -343,7 +348,7 @@ class DetectMultiBackend(nn.Module):
             binding_addrs = OrderedDict((n, d.ptr) for n, d in bindings.items())
             context = model.create_execution_context()
             batch_size = bindings['images'].shape[0]
-        else:  # TensorFlow model (TFLite, pb, saved_model)
+        else:  # TensorFlow (TFLite, pb, saved_model)
             if pb:  # https://www.tensorflow.org/guide/migrate#a_graphpb_or_graphpbtxt
                 LOGGER.info(f'Loading {w} for TensorFlow *.pb inference...')
                 import tensorflow as tf
@@ -425,6 +430,7 @@ class DetectMultiBackend(nn.Module):
             y[..., 1] *= h  # y
             y[..., 2] *= w  # w
             y[..., 3] *= h  # h
+
         y = torch.tensor(y) if isinstance(y, np.ndarray) else y
         return (y, []) if val else y