ABSYZ ABSYZ

  • Home

    Home

  • About us

    Who We Are

  • Our Expertise

    What we Do

  • Our Approach

    How We Do It

  • Products

    What We Made

  • Industries

    Who We Do It For

  • Clients

    Whom We Did It For.

  • Article & Blogs

    What Experts Think

  • Careers

    Join The Team

  • Get In Touch

    Let’s Get Started

ABSYZ

Grocery Stock Maintenance Using Einstein Object Detection

Home / Article & Blogs / Apex / Grocery Stock Maintenance Using Einstein Object Detection
By Pushmitha Babu inApex, Lightning, Salesforce, Salesforce Einstein

Let suppose a grocery store wanted to update the daily stock of juice cartons that are of different brand refrigerated. This cannot be done manually every day by counting each carton of a different brand. Think when you have a system that helps you in identifying daily stock, here Einstein Object Detection helps the grocery owner. Einstein identifies the brand and boundaries from which we can categorize them. From the data, we can make a count of different brand cartons and generate a report to the grocery owner.

Einstein should identify objects so first, we should train with the dataset. To create a dataset, collect combinations of brand images. Here I have taken 4 brands of juice cartons as Real, Tropicana, Natural, and Nescafe. The images in dataset should be like the permutation of these cartons.

obj1

Make a folder saving all these images to it and create a .csv file inside the folder. The .csv should contain height, width, x-axis, and y-axis of each carton. The format in the .csv file should be as specified: Box1 {“height”:494,”Y”:410,”label”:”Tropicana”,”width”:284,”x”:11}. The image name, for example, juice1.jpg should be specified in the first column of the sheet and the corresponding boundaries in the above format. Refer below images to create the .csv file.

In the final folder, all images and .csv file are saved. Zip the folder and upload it to AWS to get downloadable link to train the model. To know now how to upload in AWS and get the link refer (https://blogs.absyz.com/2018/02/13/einstein-vision-real-estate-app/).

Using the link we have to train Einstein

obj7In the Einstein package, modify some lines of code in the Einstein Prediction Service class.

[sourcecode language=”java”]
private static String BASE_URL = ‘https://api.einstein.ai/v2’;
private static String BASE_URL = ‘https://api.einstein.ai/v2’;
private String PREDICT = BASE_URL + ‘/vision/detect’;
//all through code the url of httpbodypart should be image-detection
EinsteinVision_HttpBodyPartDatasetUrl parts = new EinsteinVision_HttpBodyPartDatasetUrl(url,’image-detection’);
[/sourcecode]

To get the output with the predicted boundaries in the probability class add this below code.

[sourcecode language=”java”]
@AuraEnabled
public BoundingBox boundingBox {get; set;}
public class BoundingBox {
@AuraEnabled
public Integer minX {get; set;}
@AuraEnabled
public Integer minY {get; set;}
@AuraEnabled
public Integer maxX {get; set;}
@AuraEnabled
public Integer maxY {get; set;}
}
[/sourcecode]

In this apex class, a wrapper is created for the image and record. I have to display the input image to the user hence wrapper is used.  The method that returns the predicted value.

[sourcecode language=”java”]
@AuraEnabled
public static objects__c getPrediction(id objectId,String fileName,String base64) {
wrapperClass returnwrapperClass = new wrapperClass ();
objects__c obj = new objects__c();
Blob fileBlob = EncodingUtil.base64Decode(base64);
EinsteinVision_PredictionService service = new EinsteinVision_PredictionService();
EinsteinVision_Dataset[] datasets = service.getDatasets();
List<ContentDocument> documents = new List<ContentDocument>();
for (EinsteinVision_Dataset dataset : datasets) {
if (dataset.Name.equals(‘juice’)) {
EinsteinVision_Model[] models = service.getModels(dataset);
EinsteinVision_Model model = models.get(0);
EinsteinVision_PredictionResult result = service.predictBlob(model.modelId,fileBlob, ”);
EinsteinVision_Probability probability = result.probabilities.get(0);
string resultedProbablity = ”;
Map<string,integer> items = new Map<string,integer>();
for(integer i=0;i<result.probabilities.size();i++){
if(!items.containskey(result.probabilities.get(i).label)){
items.put(result.probabilities.get(i).label,1);
}
else{
integer count = items.get(result.probabilities.get(i).label);
items.put(result.probabilities.get(i).label,count+1);
}
integer j = i+1;
}
for(String i : items.keyset()){
resultedProbablity = resultedProbablity +’ ‘+i+’ — ‘+’ ‘+items.get(i);
}
obj = [select id,Results__c from objects__c where id =: objectId];
obj.Results__c = resultedProbablity;
update obj;
returnwrapperClass.objectRecord = obj;
ContentVersion contentVersion = new ContentVersion(
Title = fileName,
PathOnClient = fileName+’.jpg’,
VersionData = fileBlob,
IsMajorVersion = true
);
insert contentVersion;
documents = [SELECT Id, Title, LatestPublishedVersionId,createddate FROM ContentDocument order by createddate desc];
//create ContentDocumentLink record
ContentDocumentLink cdl = New ContentDocumentLink();
cdl.LinkedEntityId = objectId;
cdl.ContentDocumentId = documents[0].Id;
cdl.shareType = ‘V’;
insert cdl;
}
}
return obj;
}
[/sourcecode]

Here I have created a component that is accessed from mobile using salesforce one app. Firstly we have to send an image to Einstein so we take a photo through mobile. Second, we have to upload the image to Einstein. Finally, we get the output that can be displayed and further processed to create monthly reports on the stock and sales data.

[sourcecode language=”html”]
<aura:component implements=”force:appHostable,flexipage:availableForAllPageTypes,force:hasRecordId” access=”global” controller=”EinsteinVision_Admin”>
<aura:attribute name=”contents” type=”object” />
<aura:attribute name=”Objectdetection” type=”objects__c” />
<aura:attribute name=”files” type=”Object[]”/>
<aura:attribute name=”image” type=”String” />
<aura:attribute name=”recordId” type=”Id” />
<aura:attribute name=”newPicShow” type=”boolean” default=”false” />
<aura:attribute name=”wrapperList” type=”object”/>

<lightning:card iconName=”standard:event” title=”Object Detection”>
<aura:set attribute=”actions”>
<lightning:button class=”slds-float_left” variant=”brand” label=”Upload File” onclick=”{! c.handleClick }” />
</aura:set>
</lightning:card>
<aura:if isTrue=”{!v.newPicShow}”>
<div style=”font-size:20px;”>
<h1>Result1 : {!v.Objectdetection.Results__c}</h1>
</div>
<div class=”slds-float_left” style =”height:500px;width:400px”>
<img src=”{!v.image}”/>
</div>
</aura:if>
<div>
<div aura:id=”changeIt” class=”change”>
<div class=”slds-m-around–xx-large”>
<div role=”dialog” tabindex=”-1″ aria-labelledby=”header99″ class=”slds-modal slds-fade-in-open “>
<div class=”slds-modal__container”>
<div class=”slds-modal__header”>Upload Files
<lightning:buttonIcon class=”slds-button slds-modal__close slds-button–icon-inverse” iconName=”utility:close” variant=”bare” onclick=”{!c.closeModal}” alternativeText=”Close window.” size=”medium”/>
</div>
<div class=”slds-modal__content slds-p-around–medium”>
<div class=” slds-box”>
<div class=”slds-grid slds-wrap”>
<lightning:input aura:id=”fileInput” type=”file” name=”file” multiple=”false” accept=”image/*;capture=camera” files=”{!v.files}” onchange=”{! c.onReadImage }”
label=”Upload an image:”/>
</div>
</div>
</div>
<div class=”slds-modal__footer”>
</div>
</div>
</div>
</div>
</div>
</div>
</aura:component>
[/sourcecode]

In the controller.js, handling the user input and passing it to apex in return probability with boundaries as resultant.

[sourcecode language=”javascript”]
({
onUploadImage: function(component, file, base64Data) {
var action = component.get(“c.getPrediction”);
var objectId = component.get(“v.recordId”);
action.setParams({
objectId: objectId,
fileName: file.name,
base64: base64Data
});
action.setCallback(this, function(a) {
var state = a.getState();
if (state === ‘ERROR’) {
console.log(a.getError());
} else {
component.set(“v.Objectdetection”, a.getReturnValue());
var cmpTarget1 = component.find(‘changeIt’);
$A.util.addClass(cmpTarget1, ‘change’);
component.set(“v.newPicShow”,true);
}
});
},
onGetImageUrl: function(component, file, base64Data) {
var action = component.get(“c.getImageUrlFromAttachment”);
var catId = component.get(“v.recordId”);
action.setParams({
objId: objId
});
action.setCallback(this, function(a) {
var state = a.getState();
if (state === ‘ERROR’) {
console.log(a.getError());
} else {
if (!a.getReturnValue()==”) {
component.set(“v.image”, “/servlet/servlet.FileDownload?file=” + a.getReturnValue());
}
}
});
$A.enqueueAction(action);
}
})
[/sourcecode]

helper.js

[sourcecode language=”javascript”]
({
onUploadImage: function(component, file, base64Data) {
var action = component.get(“c.getPrediction”);
var objectId = component.get(“v.recordId”);
action.setParams({
objectId: objectId,
fileName: file.name,
base64: base64Data
});
action.setCallback(this, function(a) {
var state = a.getState();
if (state === ‘ERROR’) {
console.log(a.getError());
alert(“An error has occurred”);
} else {
component.set(“v.Objectdetection”, a.getReturnValue());
var cmpTarget1 = component.find(‘changeIt’);
$A.util.addClass(cmpTarget1, ‘change’);
component.set(“v.newPicShow”,true);
}
});
}
})
[/sourcecode]

The final output is tested from the mobile salesforce one app.

In case of any doubts feel free to reach out to us.

Create DatasetEinsteinEinstein Object DetectionObject Detection
142
Like this post
11 Posts
Pushmitha Babu

Search Posts

Archives

Categories

Recent posts

BioAsia 2023 in Hyderabad: An Annual International Event

BioAsia 2023 in Hyderabad: An Annual International Event

The Role Of Marketing in Small & Medium Enterprises

The Role Of Marketing in Small & Medium Enterprises

Salesforce For Retail: How Salesforce CRM Can Help Retailers

Salesforce For Retail: How Salesforce CRM Can Help Retailers

What is ChatGPT & How Does It Work?

What is ChatGPT & How Does It Work?

What Is Graphic Design? (Executive Summary 2023)

What Is Graphic Design? (Executive Summary 2023)

  • Previous PostHandling Exceptions in Lightning using AuraHandledExceptions
  • Next PostGetting started with Salesforce DX

Related Posts

Salesforce For Retail: How Salesforce CRM Can Help Retailers
Salesforce

Salesforce For Retail: How Salesforce CRM Can Help Retailers

Introduction To Copado Devops Tool
Salesforce

Introduction To Copado Devops Tool

What is Salesforce Code Builder?
Salesforce

What is Salesforce Code Builder?

Automation in Healthcare And Its Benefits
Health Cloud Salesforce

Automation in Healthcare And Its Benefits

3 Comments

  1. Bharath
    Reply
    29 May 2018

    Creative!!!!

    Reply
  2. Rohith H
    Reply
    31 May 2018

    Awesome

    Reply
  3. vamshi
    Reply
    8 October 2019

    Great Learning!!

    Reply

Leave a Reply (Cancel reply)

Your email address will not be published. Required fields are marked *

*
*

ABSYZ Logo

INDIA | USA | UAE

  • About us
  • Article & Blogs
  • Careers
  • Get In Touch
  • Our Expertise
  • Our Approach
  • Products
  • Industries
  • Clients
  • White Papers

Copyright ©2022 Absyz Inc. All Rights Reserved.

youngsoft
Copy
We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “ACCEPT ALL”, you consent to the use of ALL the cookies. However, you may visit "Cookie Settings" to provide a controlled consent. Privacy Policy
Cookie SettingsREJECT ALLACCEPT ALL
Manage consent

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary
Always Enabled

Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.

CookieDurationDescription
cookielawinfo-checkbox-analytics11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional11 monthsThe cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy11 monthsThe cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.

Functional

Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.

Performance

Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.

Analytics

Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.

Advertisement

Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.

Others

Other uncategorized cookies are those that are being analyzed and have not been classified into a category as yet.

SAVE & ACCEPT