Fast Forward: Camera As A Platform Takes The Spotlight At Facebook F8 2017

Editor’s Note: As a general version of our actionable intelligence products, this version only includes generic suggestions for brands. For industry-specific versions, please contact our Client Services Director Samantha Holland ([email protected]).

The highlights:

Facebook stakes a claim in mobile AR with Camera Effects Platform
• Messenger Platform updated with better discovery and smarter bots
• Social VR generates new use cases for 360-degree content

On Tuesday, Facebook kicked off its annual F8 developer conference in San Jose, where the social media giant unveiled several new initiatives aimed at ushering in the next stage of its mobile empire via camera-enabled augmented reality, social VR, and new AI-powered Messenger experiences. The Lab was on the ground at this year’s event to attend developer sessions and bring you all the things that marketers need to know about Facebook’s announcements this year.

Facebook Goes All-In On Camera-Enabled Mobile AR

The biggest takeaway from this year’s F8 is that Facebook is prioritizing mobile AR in its development plan and giving Snapchat a run for its money. Facebook has been busy adding “Snapchat-clone” AR features across its mobile apps during the past year or so. However, for mobile AR to truly go mainstream, it needs to evolve beyond wacky selfie lenses and start providing real value for users. And that requires an open platform that developers can leverage to create new interactive experiences. This is where Facebook’s new Camera Effects platform comes in.

Facebook is rolling out this Camera Effects developer platform in two stages. The first one, Frame Studio, is live now and accessible to anyone with a Facebook account. You can use it to create static overlaid image frames for photos and videos shared on Facebook by simply uploading a still image, no coding required. The frames created will start appearing in the Facebook Camera for your friends or fans of a Page in May. The next stage is AR Studio, which enables designers and developers to create more sophisticated AR camera effects and interactive experiences in Facebook Camera. It is currently in closed beta, and Facebook says some AR effects will become available to public in June.

With AR Studio, designers will be able to use Photoshop-like tools to create animated masks and lenses which automatically map to facial features, such as crowns and animal faces. More AI-powered tools such as SLAM (Simultaneous Localization and Mapping) positioning and object recognition are coming later this year, which will allow developers to tie AR effects to specific objects or locations visible within the frame. In the future, users will also be able to add augmented reality notes to specific real-world locations, so they could, for example, tell friends their favorite dish at a restaurant.

The AR effects also won’t be one-size-fits-all, as developers using AR Studio will be able to customize those AR camera effects with data from the network, such as sports scores, weather, and travel data. They can also access data from the mobile device’s sensors, such as GPS location, orientation, speed, and even face/gesture recognition to create more personalized user experiences. In a clever touch, Facebook added call-to-action “Try It” buttons to appear alongside the AR effects to encourage user sampling and viral spreading. Creators will also be able to target their creations according to location, day and time, and all the standard demographic targeting from Facebook’s ad platform. For now, this targeting comes at no charge to the creator.

With the launch of this camera-based AR platform, Facebook is staking a claim in AR, building the platform and tools to create interactive experiences which use the camera as an input. Right now, that’s limited to the Facebook app on our smartphones, but Zuckerberg did mention “AR glasses” a couple of times in his opening keynote address, and Facebook will be in a good position to capitalize on glasses (or any other AR, like car windshields) with this developer platform, which allows developers to “code against the real world” and unleash a tidal wave of crowd-sourced creative to overtake Snapchat’s Lenses, which are all currently built in-house.

Facebook Camera EffectsFacebook F8 AR Studio

Messenger Platform Gets Smarter Thanks To A.I. And M Integrations

Besides the shiny new AR developer platform, Facebook is also renewing its commitment to building out the Messenger Platform with a major upgrade aimed at making its messaging app smarter and more brand-friendly. The Messenger Platform 2.0 Facebook launched on Tuesday comes with new tabs for businesses and bots, offline support for parametric QR codes, as well as new Chat Extensions that can provide new experiences in group chats.

With 200 million messages sent between businesses and users since last year and 100 thousand bots already on the platform, Facebook is already the “white pages” of the internet. Now it wants to also become the “yellow pages” with the upcoming addition of a Discover tab dedicated to chatbots, making it easier for users to discover new bots that they may find interesting or useful. Messenger users can tap the hexagon icon on the bottom right and see a list of Featured businesses along with popular bots, what’s nearby, and a variety of categories like Food & Drink, Lifestyle, and News.

To further aid bot discovery, Facebook updated the parametric QR Messenger codes to allow users to quickly connect with a bot (say at an event or a concert) just by scanning the codes with their camera in Messenger. Multiple codes can be generated for the same bot so as to offer some context such as locations and analytical data for the bot, which then may offer a more tailored response accordingly – for example, deep linking to information about a specific location, inside of a bot which covers a national retail chain. This also gives users a reason to scan even if they’re already connected to the bot.

Prior to F8, Facebook had already started to roll out its AI-powered virtual assistant M on Messenger to offer automated suggestions such as calling an Uber or looking up local restaurants based on the conversations. At this week’s event, they announced the first way for third parties to integrate with M using the new Chat Extensions functionality. Chat Extensions allow a more graphical interface to open within the message thread. For instance, Facebook announced on stage a Spotify integration that will allow people to easily search and share music with each other, without ever leaving Messenger. They work in a similar way as Apple’s iMessage apps, where you can pull up an applet to access a developer-controlled experience within Messenger, and then share the results of that back to your chat. Over time, M will learn to recognize a task a user is trying to accomplish, and recommend Chat Extensions which may be able to help.

Facebook is also working to make its messaging system smarter and more responsive by launching Smart Replies for small businesses that don’t necessarily have the resources to build a full-on chatbot but still want an easy way to communicate with customers. This new AI-enabled feature will allow businesses to enable automated answers to frequently asked questions, thus simplifying the customer service on Facebook Pages and Messenger. Smart Replies are currently being trialed with U.S. restaurants, with information populated directly from their websites, no data entry or setup required.

Facebook Messenger Chat Extensions Facebook Messenger Parametric QR Codes

Social VR Generates New Use Case For 360-Degree Content

Looking beyond the camera and Messenger, Facebook is also taking a big step in virtual reality with the launch of Facebook Spaces, marking the company’s first significant integration of virtual reality into its core social product. It will run on Oculus Rift with the Touch controllers to bring social interactions into 3D virtual spaces, allowing users to generate personalized avatars and hang out together in virtual reality.

Unlike most online social platforms, the communication in Facebook Spaces won’t be text-oriented, but instead will rely on voice, facial expressions, and body language. In a demo, Facebook’s photo recognition AI helps users to create animated avatars of themselves, who can then hang out together in both virtual spaces as well as real world settings based on 360-degree photos and videos. Facebook describes it as a “virtual dinner party,” where users can talk, create 3D objects, explore 360 videos, and even take in-VR selfies.

Facebook has indicated that although Spaces is designed to be a home screen for VR, third party integrations won’t appear in the forms of VR apps, but rather as environments and objects inside the virtual space.

In addition, Facebook Spaces can use Messenger to place video calls to people out in real world, which is an interesting distribution strategy to spread awareness for this new product. People who receive video calls from Facebook Spaces will show up in the virtual space on a flat screen as normal video calls would appear, but they will be able to see and talk with their friends’ virtual avatars.

Facebook Spaces Social VR

What Brands Need To Do

Altogether, what Facebook announced on Tuesday are indicative of how the world’s biggest online audience aggregator is going to evolve, and it is up to brands to keep up with this evolution into mobile AR and social VR. The launch of the Camera Effects platform holds great potential for brands, as it allows them to work with developers to create AR camera effects to engage mobile users and, in some cases, pull in personal and contextual data to create unique experiences that they will want to share.

Facebook showcased some AR effects from its brand partners on stage, including a Nike headband overlay that featured bragging right stats from your recent run and an overlaid map, a Mass Effect-themed helmet mask that pulls in stats from your game progress, as well as a video lens created by Manchester United that pulls in real-time soccer match scores, completed with “GOAL!” and falling confetti overlays.

For now, however, there are currently no paid placement opportunities. The AR effects will be surfaced via a ranking algorithm within the camera itself, based on attributes such as location, date/time, and keywords, as well as how likely they think an individual is to share according to their past behaviors. But the “Try It” call-to-actions on shared media is an incentive to create great lenses, as it will help them earn organic distribution on the platform.

Brands looking to stay ahead of the technological curve need to start developing a AR strategy today to organically insert their brands directly into the camera and therefore into the photos and videos that people share on Facebook. While the current lenses appear similar to Snapchat, the open developer platform means that Facebook’s AR lenses will advance quickly, and are available to anyone with the designers and developers to create compelling experiences.

For CPG brands, this could mean creating fun, seasonal frames or AR lenses to remind consumers of your products and target users based on Facebook’s data, whereas healthcare and fitness brands can take some inspiration from the Nike headband lens Facebook demoed and start thinking about how to create similar AR camera effects that can amplify and enrich a great health and fitness experience for their customers.

With Messenger Platform 2.0., brands will have better tools to create smarter chatbots to connect with the over 1.2 billion active users on Messenger. Brands should consider creating various physical call-to-actions with the new parametric QR Messenger codes linking to their chatbots within different contexts, thus creating convenient chat experiences for users by deep-link them to specific functions within the bots. For restaurants and QSR brands, this could creating different QR codes for different restaurant locations and leverage location data to provide a localized experience for the customers when they turn to chatbots for information or directions.

In addition, brands also need to start thinking about ways to use Chat Extension to create experiences within Messenger that don’t fit in a bot/conversational interface. For retailers, this could mean creating an extension similar to what is doing to reach shoppers in chats.

While the beta launch of Facebook Spaces is not integrated with Facebook’s main platform and therefore limited in its opportunities for brand integrations, this is only the starting point for social VR, and we are confident that brands will have plenty of ways to get into the virtual social spaces as Facebook continues to build out this platform.

As many of the virtual environments in Facebook Spaces are generated from 360-degree photos and videos, pulled right from a user’s Facebook Photos, it’s a great incentive for brands to start investing in 360 content now, and getting it in the hands of their customers. Sharing that content in Spaces could be a great way to enable existing fans to advocate for your destination, hotel, car, or entertainment experience.

How We Can Help

While mobile AR and social VR are still in early stages of development, brands greatly benefit by starting to develop strategies for these two emerging areas. If you’re not sure where to start, the Lab is at your service.

The Lab has always been fascinated by the enormous potential of AR and its ability to transform our physical world. If you haven’t read our analysis on the Pokémon Go phenomenon last summer, which introduced AR technology to mainstream consumers, you can check it out here for more AR-related brand suggestions. We’re excited at having an open development platform from Facebook, allowing us to develop mobile AR experiences that can scale to reach billions of people. If you’d like to discuss more about how your brand can properly harness the power of mobile AR to engage your customers and create extra value, please reach out and get in touch with us.

We also have extensive experience in building AI-powered chatbots to reach consumers on messaging apps. The Niro bot we built in collaboration with Ansible for Kia is a good example of our chatbot expertise. To help our clients create an optimized conversational experience, we developed Dialogue, a dedicated conversational practice backed by our stack of best-in-class technology partners and an insights engine that extracts business intelligence from conversational data.

As for VR, our dedicated team of experts is here to guide marketers through the distribution landscape. We work closely with brands to develop sustainable VR content strategies to promote branded VR and 360 video content across various apps and platforms. With our proprietary technology stack powered by a combination of best-in-class VR partners, we offer customized solutions for distributing and measuring branded VR content that truly enhance brand messaging and contribute to the campaign objectives.

If you’d like to know how the Lab can help your brand figure out how to tap into these tech trends manifested at F8 to supercharge your marketing efforts, please contact our Client Services Director Samantha Holland ([email protected]) to schedule a visit to the Lab.

What To Expect From This Year’s Facebook F8 Event

What’s Happening
Facebook’s annual developer conference F8 will take place this Tuesday and Wednesday in San Jose, where the company is expected to announce updates and lay out its plan for some of its key initiatives such as messaging, artificial intelligence, and live video. This year’s event will kick off with a keynote from CEO Mark Zuckerberg on Tuesday morning, and will presumably cover most of those key areas as before, possibly with some additional discussions on Facebook’s role in combating fake news and promoting global connectivity.

At the F8 event last year, Facebook launched its bot platform for Messenger, demoed a more “social” virtual reality with Oculus, and rolled out a 360-degree video camera, and we expect to see updates in all three areas this year as well. In fact, Facebook has already unveiled several things that will be covered at this year’s event, including bots for Messenger groups, offline Instagram, and a new Camera Effects Platform allowing developers to create more features for the Facebook camera, both in its flagship app and Messenger.

What Brands Need To Do
Facebook and its various properties and apps are still unparalleled in their global reach and utterly dominating in terms of mobile time spent. Facebook Messenger just passed over 1.2 billion monthly active users this week. Its aggressive cloning of Snapchat’s features has apparently started to pay off, with its Instagram Stories surpassing Snapchat in monthly active user earlier this month. The new Camera Effects platform also hints at Facebook’s ambition in pushing deeper into selfie lenses and other mobile AR features. Moreover, Facebook Live also presents a strong area of media growth, as the company announced earlier this month that live video now account for 20% of video on the platform, with the number of broadcasts grown more than four times over the last year.

This year, the Lab has a special team attending F8 to bring us the latest announcements. You can follow us on Twitter @ipglab for live updates. Also, remember to check back for our in-depth take on Facebook’s new products and services and how they would impact the future of brand marketing.


Source: TechCrunch

Fast Forward: What Marketers Need To Know About Facebook’s F8 Event

Your guide to tech-driven changes in the media landscape by IPG Media Lab. A fast read for you and a forward for your clients and team.

• Facebook launched Messenger Platform to help brands to communicate with users
• Live Video with API strengthens video and video ads on Facebook
• New tools for app owners and VR content creators

Screen Shot 2016-04-12 at 1.10.50 PM

What Facebook Announced
At its annual developer conference on Wednesday, Facebook discussed their ten-year roadmap and introduced a number of new features to improve its platform and make it easier for brands to connect with Facebook users. Following Microsoft’s launch of a bot framework two weeks ago, Facebook is introducing chat bots and developer tools to its messaging app with the debut of Messenger Platform. During his opening keynote, Facebook CEO Mark Zuckerberg demoed chat bots from CNN and 1-800-Flowers to showcase how chat bots can help publishers and businesses communicate with users on Messenger. While there will not be a curated bot store, the Messenger team does promise to help with discovery, mostly via search.

Facebook also introduced a Live API so that developers can live stream video to Facebook from any app or device. The new API should vastly expand the reach of Facebook’s Live video broadcasts as it allows lots of new broadcasting methods, like the use of multiple cameras, drones, and new devices. This is in sharp contrast to Facebook’s biggest live-stream competitor, Twitter’s Periscope, which requires streaming from a single smartphone. Therefore, FB Live can scale from a front-facing smartphone camera through TV production quality equipment and has already shown it can draw audiences that rival cable TV with the right content.

Moreover, Facebook is diving deeper into virtual reality with an open-source Surround 360 camera rig to help professional content creators capture 360-degree video with minimal post-processing. It is also looking to ship the Touch controllers for Oculus VR later this year to add interactivity to virtual reality. Zuckerberg also teased some multiplayer features that Facebook is working on to make virtual reality a more social experience.


Facebook continues to build out its ecosystem with to two new tools: Account Kit and a new “Save to Facebook” button. With Account Kit, developers can streamline account signup and login, extending the popular “Login with Facebook” feature to users without Facebook accounts. Facebook is also expanding the four-year-old “Save for Later” feature with a new “Save to Facebook” button, which enables users to save content from third-party sites to read or watch later in the Saved tab Facebook.

Facebook also shared some interesting stats on its fast-loading Instant Articles program. Since launch, Facebook says posts in Instant Articles have logged 20% more opens and a 70% lower abandonment rate than other content. They also report an average of 30 seconds of engagement on Canvas Ads.

What Brands Need To Do

• Develop useful bots to reach customers on the messaging apps they’re already using
• Take advantage of the emerging popularity of live streaming to connect with consumers in real time
• Start producing branded VR and 360 video content to immersively engage consumers

As Messenger hits 900 million monthly users, there is no denying that it is important for brands to be present on Facebook’s messaging platform to connect with those users. With the launch of Messenger Platform, Facebook is making it possible for brands to build bots to communicate and serve customers directly on Messenger. For example, Facebook showcased a shopping assistant bot from Spring, which suggests appropriate items for shoppers to purchase directly on Messenger. Facebook’s demo shows that chat bots can be trained with basic responses on a web interface with no code while more in-depth responses require custom development, aided by Facebook’s new Bot Engine tools. Particularly interesting here is the Customer Matching feature, which allows brands to sync up customer profiles and conversations across Messenger and SMS.


With the Live API, Facebook is making it easier for brands and developers to integrate its relatively new live-streaming feature into their apps, platforms, and even devices. As we noted recently, live-streaming video is starting to blow up thanks to Facebook. Live video content lends an authentic touch to brands, which millennials seem to embrace. For brands looking to experiment with live-streaming, they should be prepared on the technical front to ensure a smooth broadcast, but also willing to roll with the punches and embrace the unexpected.

As Facebook makes a strong push for VR and 360-degree videos, brands would be smart to seize the opportunity and work closely with content creators to craft interactive VR experiences to intrigue and engage consumers. The new Facebook Surround 360 camera should be a great tool for brands to produce 360-degree content, and the coming interactive and social features should help make virtual reality a more powerful tool for brands to create immersive branded content to engage their audiences with.

How We Can Help
The Lab is actively producing 360 content, chat bots, and live social video products for our clients. Please contact Client Services Director Samantha Holland ([email protected]) if you would like more detail or to schedule a visit to the Lab to discuss how your brand can better serve your customers with these emerging platforms.

For previous editions of Fast Forward, please visit Please reply with any constructive criticism or feedback. We want these to be as useful as possible for you and your clients, and your input will help us immensely.


All pictures featured courtesy of the F8 Livestream