Our system is capable of transcribing any type of unseen circular analogue gauge. It is robust to lighting, camera angle, can correct for the parallax effect and can run in real-time on mobile.
We use an interpretable methodology to extract a gauge reading, built with a full end-to-end deep learning pipeline. Two convolutional neural networks (CNN's) were used and were trained solely with synthetic data, this dataset has been made publicly available. The system was evaluated on a real image dataset, also made publicly available, and beats current state-of-the art gauge transcription systems in multiple metrics by large margings.
Please see below for a demo of our system running on iPad pro at 100 fps in real time.
We also have our presentation at CVPR 2021 MAI workshop.
Two datasets have been released, (1) SyntheticGauges, a dataset of synthetically rendered gauges aimed at training CNN's, (2) RealGauges, a real image gauge dataset aimed at evaluating three core elements of gauge transcription systems; gauge detection, pose recovery and gauge reading.
Both the SyntheticGauges and RealGauges datasets can be downloaded from here: Download
The SyntheticGauges dataset is split into a 10,000 image train set and a 1,000 image test set. Both sets are made up of images with a 1024x1024px resolution and are accompanied by json files which contain ground truth labels. Ground truth labels describe the bounding box of the gauge along with keypoints for the perspective points and the scale minimum, scale maximum, pointer center and pointer tip (read the paper for details), they are in COCO format.
The RealGauges dataset is formed from 6 different gauges and evaluates three different tasks. For gauge detection, each gauge is photographed on 36 different backgrounds from an Interior Design magazine, the center of each gauge is labelled. For pose recovery each gauge is photographed from a variety of camera angles, the normal to the gauge plane is annotated to evaluate pose revovery performance. For gauge reading each gauge is captured in 3 distinct 5 second videos with various modes of pointer motion, the reading of the gauge and the angle of the pointer relative to the scale minimum is labelled.
LICENSE
(c) Ben Howells, James Charles and Roberto Cipolla. Department of Engineering, University of Cambridge 2020
By downloading this dataset, you agree to the Creative Commons Attribution-NonCommercial 4.0 International license. This license allows users to use, share and adapt the dataset, so long as credit is given to the authors (e.g. by citation) and the dataset is not used for any commercial purposes.
THIS SOFTWARE AND ANNOTATIONS ARE PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.