Using Artificial Intelligence with Azure Cognitive Services

by Oct 2, 2018

The cloud these days not only offers virtual machines and storage, but also brand new and exciting services such as the cognitive services. You need an Azure subscription key (which you can get for free on the website mentioned below) if you want to directly access these services. Else, you would need to resort to the free interactive demos found here:

https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/#analyze

Now here’s a script that sends an image file to the Azure image analysis, and you get back a detailed description of what is seen on the picture, including coordinates of faces, gender, and estimated age:

# MAKE SURE YOU SPECIFY YOUR FREE OR PAID AZURE SUBSCRIPTION ID HERE:
$subscription = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"


Add-Type -AssemblyName System.Net.Http
# MAKE SURE YOU SPECIFY A PATH TO A VALID IMAGE ON YOUR COMPUTER
$image = "C:\sadjoey.jpg"
$uriBase = "https://westcentralus.api.cognitive.microsoft.com/vision/v2.0/analyze"
$requestParameters = "visualFeatures=Categories,Tags,Faces,ImageType,Description,Color,Adult"
$uri = $uriBase + "?" + $requestParameters

# get image file as a byte array
$imageData = Get-Content $image -Encoding Byte 

# wrap image into byte array content
$content = [System.Net.Http.ByteArrayContent]::new($imageData)
$content.Headers.ContentType = [System.Net.Http.Headers.MediaTypeHeaderValue]::new("application/octet-stream")

# get a webclient ready
$webclient = [System.Net.Http.HttpClient]::new()
$webclient.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key",$subscription)

# post the request to Azure Cognitive Services
$response = $webclient.PostAsync($uri, $content).Result
$result = $response.Content.ReadAsStringAsync().Result

# convert information from JSON into objects
$data = $result | ConvertFrom-Json

# get image detail information
$data.description.captions
$data.Faces | Out-String
$data.description.Tags

The result looks similar to this:

 
PS C:\> $data.description.captions

text                                          confidence
----                                          ----------
a man and a woman standing in a room 0,94136500176759652



PS C:\> $data.faces

age gender faceRectangle                            
--- ------ -------------                            
 23 Female @{top=114; left=229; width=47; height=47}



PS C:\> $data.description.tags
person
indoor
man
holding
woman
standing
window
table
room
front
living
young
video
computer
kitchen
playing
remote
wii
people
white
game 
 

You can also take a look at the original JSON data returned by the web service:

 
$result
{"categories":[{"name":"indoor_","score":0.42578125},{"name":"others_","score":0
.00390625}],"tags":[{"name":"person","confidence":0.999438464641571},{"name":"in
door","confidence":0.99413836002349854},{"name":"wall","confidence":0.9905883073
8067627}],"description":{"tags":["person","indoor","man","holding","woman","stan
ding","window","table","room","front","living","young","video","computer","kitch
en","playing","remote","wii","people","white","game"],"captions":[{"text":"a man
 and a woman standing in a room","confidence":0.94136500176759652}]},"faces":[{"
age":23,"gender":"Female","faceRectangle":{"top":114,"left":229,"width":47,"heig
ht":47}}],"adult":{"isAdultContent":false,"adultScore":0.023264557123184204,"isR
acyContent":false,"racyScore":0.042826898396015167},"color":{"dominantColorForeg
round":"Brown","dominantColorBackground":"Black","dominantColors":["Brown","Blac
k","Grey"],"accentColor":"6E432C","isBwImg":false},"imageType":{"clipArtType":0,
"lineDrawingType":0},"requestId":"8aebed85-68eb-4b9f-b6f9-5243cd20e4d7","metadat
a":{"height":306,"width":408,"format":"Jpeg"}}  
 

Twitter This Tip! ReTweet this Tip!