Getty Images-Connect-API-Hackathon

Leveraging our Connect API? Oh the things you can do!!!

May 17, 2012 | By Karthik Srinivasan | By Devs, For Devs, Technology

Editor’s note: The TechCrunch ‘Disrupt NY’ Hackathon is set for May 19-20, and we’re thrilled to have our developers on site to facilitate. Equally thrilling, Disrupt participants will be able to use our API in their hacks, and we can’t wait to see what they come up with. Until then, here are a few insights from Getty Images Software Engineer Karthik Srinivasan, whose team won an internal Hackathon we held recently in our Getty Images offices.

Anne-Hathaway-Ethan-Miller-Getty-Images-109701608

Actress Anne Hathaway arrives at the 83rd Annual Academy Awards at the Kodak Theatre February 27, 2011 in Hollywood, California. (Photo by Ethan Miller/Getty Images)

 

With the recent release of the Connect API,  Getty Images decided to have its own internal Hackathon amongst various internal teams at Getty Images.  What better way to show people the power of an API than to eat your own dog food!

Overall, 17 teams submitted entries.  The winning team, my team, submitted a project called Headhunter.  The concept behind it was to be able to search images using the Connect API and to sort/filter those images by composition values, such as face only, half-body length, full-body length, facing camera, angled face, side view, single shot, multi person shot, etc…. in an effort to find the best images of some popular celebrities people would be interested in.  I’d like to explain further how my team was able to build this project leveraging technology form both the Connect API as well as the Face.com API.

The following is a high level overview of how the application worked:

 

Search images → face.com → aggregate & store

Connect API provides three primary access points to retrieve and search images and it’s metadata:

  • SearchForImages
  • GetImageDetails
  • GetEvent

More API details at https://api.gettyimages.com/sites/default/files/GettyImagesConnectAPI.pdf

SearchForImages functionality provides us the ability to search by personalities, search phrases, keywords and provide multiple filter options to refine the results. In our case, we are searching by personalities; the following is a code snippet of the search query:

var searchQuery = new SearchForImages2RequestBody();

searchQuery.Query = new Query();

searchQuery.Query.SpecificPersons = new List<string> { “Jennifer Lopez” };

searchQuery.Query.SearchPhrase = “topix”;

searchQuery.ResultOptions = new ResultOptions();

searchQuery.ResultOptions.ItemCount = 5;

 

Note:topix” is a keyword that is used to return back all images that are either top pics or image rank 1.

Other search filters that SearchForImages accepts are as follows:

  • EditorialSegments: news, sports, entertainment, publicity, royalty, archival
  • File Types: eps, jpg
  • GraphicStyles: photography, illustration
  • ImageFamilies: creative, editorial
  • LicensingModels: royaltyfree, rightsmanaged
  • Orientations: horizontal, vertical, panoramichorizontal, panoramicvertical, square
  • ProductOfferings: premiumaccess, editorialsubscription

The SearchForImages access point provides some metadata information but it is limited. To get all metadata information of the image we can extract the imageId’s from the search result, which can then be used as an input for GetImageDetails access point for retrieving full image detail. Similarly, Eventids can be passed onto GetEvent to get event information that the images are associated with. The metadata information from above three calls when merged would provide in-depth information of the image.

Example calls for above steps:

//Authenticate

var session = operations.CreateSession();

var searchQuery = new SearchForImages2RequestBody();

searchQuery.Query = new Query();

searchQuery.Query.SpecificPersons = new List<string> { “Jennifer Lopez” };

searchQuery.Query.SearchPhrase = “topix”;

searchQuery.ResultOptions = new ResultOptions();

searchQuery.ResultOptions.ItemCount = 5;

 

//Search Images

var searchResponse = operations.SearchImages(searchQuery, session.TokenResponse.Token);

 

//Image Details

var imageIds = searchResponse.SearchForImagesResult.Images.Select(s => s.ImageId).ToList();

var imageDetailRequest = new GetImageDetailsRequestBody();

imageDetailRequest.ImageIds = imageIds;

var imgDetailSrchResponse = operations.GetImageDetail(imageDetailRequest, session.TokenResponse.Token);

 

//Event Detail

var eventIds = searchResponse.SearchForImagesResult.Images.SelectMany(s => s.EventIds).ToList();

var geteventRequest = new GetEventsRequestBody();

geteventRequest.EventIds = eventIds;

var eventResponse = operations.GetEventInfo(geteventRequest, session.TokenResponse.Token);

 

Once we have a collection of images for individual personalities, we post the images to the face.com API to retrieve composition information.  Since face.com provides relative and numerical composition values, we translate these values to human readable data using the following conditions:

 

To Determine the angle of the face:

if (tag.yaw < 10 && tag.yaw > -10)

return “Forward looking face”;

 

if (tag.yaw < 20 && tag.yaw > -20)

return “Slightly Angled face”;

 

if (tag.yaw < 60 && tag.yaw > -60)

return “Angled face”;

 

if (tag.yaw < 90 && tag.yaw > -90)

return “Side face”;

 

To Determine if the image is a portrait image:

 

if (tag.height > 40 || tag.width > 40)

return “Headshot or Potrait”;

 

if ((tag.height < 40 && tag.height > 15) || (tag.width < 40 && tag.width > 15))

return “HalfLength body shot”;

 

else

return “FullLength body shot”;

 

Connect and Face.com data was then dumped into a MongoDb instance and we created a couple of Map/Reduce calls to group data (on celebrity name and event id, the map function) then reduced it in a couple of different ways. The main reduction goes through every Celebrity/Event combination and finds the image with only one face, a “topix” keyword (rank one) and/or the largest face (the biggest headshot). This is what powers the splash page: for every celebrity, you get a headshot.

Splash page with Personality Headshots using MapReduce data:

 

Getty Images-Connect-API-Hackathon

A screen grab of our HeadHunter hack using Connect, our Getty Images API.

 

Personality Page with various filter options (following screenshot are images with Headshot + Solo Shot + Forward looking)

 

Getty Images-Connect-API-Anne Hathaway

A screengrab of what our HeadHunter hack looked like.

 

Note: Above code to access Connect API was written using a C# library https://github.com/karthik20522/GettyConnect

More information of Face.com Metadata at http://kufli.blogspot.com/search/label/Face.Com or http://developers.face.com/docs

More posts by this author

Comments are closed.