Custom Computer Vision: Beyond Computer Vision -- Training Ai for a purpose

In the last blog, I talked about using basic features of computer vision api from Microsoft. This time, I would like to expend on what you can do if you want to have custom objects recognized from scratch. To create a custom recognition patterns, we have to "custom vision" project within cognitive services (Custom Vision). 

How does custom vision work?

Custom vision works by tagging images or objects and training the Ai to recognize certain patterns. You will need to "show" the Ai a minimum of 3 sets of images (good, bad, and disqualified). 

The following example shows a simple examples of train an Ai to recognize highway overpasses in good, or bad condition.

Sample good image"

Sample bad image"

Sample disqualified image:

Note: On disqualified image, I am choosing to use an image that might confuse the Ai. See the signage.

Once the system is trained, we can use the code below to start recognizing images. 

function processImage() {

       
        var subscriptionKey = "Insert Key";
        var predictionKey = "Insert Key";
        var trainingKey = "Insert Key";
       
        // Display the image.
        var sourceImageUrl = document.getElementById("inputImage").value;
        document.querySelector("#sourceImage").src = sourceImageUrl;
        // Show hide to make things pretty
        $("#responseTextArea").val("");
        $("#responseGood").css('visibility', 'hidden');
        $("#responseBad").css('visibility', 'hidden');
        $("#responseUnknown").css('visibility', 'hidden');
        $("#trainingButton").css('visibility', 'hidden'); 

        $.ajax({
            url: "https://southcentralus.api.cognitive.microsoft.com/customvision/v2.0/Prediction/##insertInstance##/url",

            // Request headers.
            beforeSend: function (xhrObj) {
                $("#responseTextArea").val("");
                xhrObj.setRequestHeader("Content-Type", "application/json");
                xhrObj.setRequestHeader("Prediction-Key", predictionKey);
            },
            type: "POST",
            // Request body.
            data: '{"url": ' + '"' + sourceImageUrl + '"}',
        })

            .done(function (data) {
                // Show formatted JSON on webpage.
                $("#responseTextArea").val(JSON.stringify(data, null, 2));                                $("#responseGood").css('visibility', 'visible');
                $("#responseBad").css('visibility', 'visible');
                $("#responseUnknown").css('visibility', 'visible');
                $("#trainingButton").css('visibility', 'visible'); 
                $("#responseGood").text(data.predictions[0].tagName.toString() + ": " + (data.predictions[0].probability * 100).toFixed(2).toString() + "%");
                $("#responseBad").text(data.predictions[1].tagName.toString() + ": " + (data.predictions[1].probability * 100).toFixed(2).toString() + "%");
                $("#responseUnknown").text("Uncertainty: "+(100 - ((data.predictions[0].probability * 100) + (data.predictions[1].probability * 100))).toFixed(2).toString()+"%"); 
            })

            .fail(function (jqXHR, textStatus, errorThrown) {
                // Display error message.
                var errorString = (errorThrown === "") ? "Error. " :
                    errorThrown + " (" + jqXHR.status + "): ";
                errorString += (jqXHR.responseText === "") ? "" :
                    jQuery.parseJSON(jqXHR.responseText).message;
                alert(errorString);
            });
    };

TLDR

1. Upload Images and Tag the images so that Ai knows the “Good”, “Bad”, and “Unrelated” images.
    
2. Train the Ai to “Learn” what those images mean.
    
3. Evaluate the Ai ML model for accuracy.
    
4. Repeat and compound with additional images.