Entitlement Engine with AppStore and GooglePlay — Part 1: Concepts
If you have implemented subscription services in your App, you’ve probably realized how challenging it can get to cover the different states a subscription. The different approaches that AppStore and GooglePlay has taken, certainly doesn’t help to reduce this complexity.
In this article I will try to share my rather painful experience of developing an entitlement engine, hoping to help you to have less headache implementing it.
This article is a series of three parts:
- Part 1 — Concepts: We won’t discuss any code here and focus only on the both stores’s behavior and the general architecture of our entitlement engine.
- Part 2 — Backend: Implementing the concepts using Node, Firestore and GCP Cloud Functions.
- Part 3 — Frontend: Implementing the front-end with React-Native.
The subscriptions are very different from one-time purchases. In a one-time purchase, the user attempts to buy a product, pays the price and it’s done. In subscriptions, however, the user agrees to pay a certain amount of money on, for example, every 10th of each month. Now, what happens if the user decides to cancel the subscription? or the user’s bank account simply doesn’t have sufficient balance and the user is not aware of this? What if we want to have a 14 days of trial period on our subscriptions?
Let’s dive in!
The General Idea
Let’s assume that you have an iOS and an Android App that provide some services through subscription. Let’s call them “Premium Services”. You also probably have a backend which handles your authentication, content serving and etc. Your authentication service keeps the “User”’s profile and other auth related information.
Now what we want to do, is to come up with a solution that gathers information about the user from your authentication service, the user’s subscription status from AppStore or GooglePlay and finally decides what kind of access the user should have to your premium services. Let’s call this system the “Entitlement Engine”.
This Entitlement Engine is in close contact with the stores. It get’s the latest information from them and mutates your subscribed users’ status accordingly.
The Problem
It’s been a while that the subscriptions are not only about being active and inactive. There are new features implemented by both of the stores that makes it rather complicated. Nowadays there are trial periods and grace periods.
For instance, let’s review some common scenarios that might happen with the subscriptions:
- User decides to cancel the subscription. Now, the subscription should be kept active until the next renewal date and after that, the user’s subscription status should be returned back to the basic plan.
In this case, You might want to encourage the user to re-subscribe via some cute pop-up in your app. - User might not cancel the subscription, but due to insufficient balance in the user’s bank account, the store is not able to charge. So the user enters the grace period.
For this case, you might want to inform the user about the situation so he or she can increase the card’s balance. - The store wan’t successful in charging the user’s bank account and the grace period is over now.
In this stage, you might want to limit the user’s access and give some warnings about the situation. - The user who has canceled the subscription in the past, re-subscribes to the service or even upgrades/downgrades while the subscription is active.
If you’re planning to use these features and you’re really concerned about a good user experience in your App, your platform should be able to handle the above cases, and give the user a proper feedback.
I strongly recommend to watch these videos from Apple which describes the process of creating an entitlement engine:
Ok, now let’s see how the subscriptions work on each store.
AppStore Subscriptions
Any purchase from the AppStore returns an encrypted base64 “receipt”. All information related to the customer’s current transaction as well as the previous transactions are included in this receipt.
In order to decrypt this receipt, you need to make a POST request to the verifyReceipt endpoint with the base64 encrypted string and a password which is your app’s shared secret. Because of this shared secret it’s important to keep this warning from Apple in mind:
Do not call the App Store server verifyReceipt endpoint from your app. You can’t build a trusted connection between a user’s device and the App Store directly, because you don’t control either end of that connection, which makes it susceptible to a man-in-the-middle attack.
Another tool from AppStore that might come handy is the S2S Notifications. S2S Notifications are HTTP calls that AppStore makes to your servers to notify a change in a user’s subscription status.
AppStore, expect to receive a 200 response, otherwise it’s gonna keep calling your endpoint again later.
GooglePlay Subscriptions
Purchase from the GooglePlay, follows roughly the same pattern. Except, you receive a token rather than an encrypted receipt. This is called “purchase token” and can be used to query the user’s subscription status.
In order to get the user’s latest subscription status, you need to send a GET request to Purchases:subscriptions endpoint and include purchase token, package name and the subscription id in the url.
Before making the request, you must authenticate to get an access token. Then include the received access token in your Authorization header as a bearer token. The details can be found here.
You can also receive notifications about the changes in users’ subscription status. Unlike AppStore which makes a HTTP call to your servers, GooglePlay publishes its notifications on GCP’s Pub/Sub channel.
Caveats with the Store Notifications
Both of the stores recommend not to solely rely on these notifications. As:
- It’s not guaranteed that you will receive notification for every change.
- It’s not guaranteed that what you are receiving contains the latest information about the customer’s subscription status.
In AppStore’s case for example, when the user cancels the subscription, we receive a DID_CHANGE_RENEWAL_STATUS event. In this case we should keep the user’s premium access active until the latest receipt is expired. But we will not receive any notification when the receipt is actually expired.
The Solution
Now that we know what kind of entitlement engine we need and what challenges we’re facing, we need to start designing our engine.
What our engine should do, can be simply summarized in these steps:
- Get the latest information about the user’s subscription status from the store.
- Digest the information received from the store and determine the user’s subscription status.
- Mutate the user’s entitlement by persisting the subscription data into the database.
Architecture
Our entitlement engine will follow this simple process:
Let’s call this “Subscription Validation”. Now what are the triggers and what’s the single source of truth?
The single source of truth is the only source that we use to generate our entitlement status. In this case, the single source of truth is the verified subscription data which we receive after calling the /verifyReceipt on AppStore and Purchases:subscriptions on GooglePlay.
What about triggers? Well, any potential event that might have something to do with the subscription, can be a good candidate for a trigger. For this article we will consider these triggers:
- When app sends the encrypted receipt or purchase token to be verified. Like when the user attempts to subscribe on the App.
- When AppStore S2S Notifications calls our server.
- When GooglePlay Realtime Developer Notifications publishes an event on the Pub/Sub topic.
- A scheduled task that runs every day to extract the expired receipts from our database.
This is necessary as it’s not guranteed to receive a notification for every state change.
Note: It looks tempting to dismiss the store notifications as triggers. This means that you should have a more sophisticated scheduled task to verify your receipts by simply polling all of the subscriptions on a fixed intervals. Although, this was actually the case before the S2S notifications came along, it’s not recommended now.
According to the Android API’s documentation:
Due to quota restrictions, it is not recommended to check state by polling the Google Play Developer API at regular intervals instead of leveraging Real-time developer notifications.
Let’s attach the triggers to the Subscription Validation and get our final entitlement engine architecture:
Great, now let’s see how this architecture would work over the time.
Subscription Lifecycle
The subscription lifecycle can be broken into two different phases. The “Initial Purchase” and the “State Change”.
For this part let’s assume a simple scenario:
Our user decides to subscribe and thus selects one of our plans. Now the following actions take place:
After the initial purchase, for example, two months later, the user decides to cancel the subscription. In this case the following actions will take place:
Now that we have nice general picture of how things should work, let’s go a bit deeper and talk about the details.
Subscription States
In order to have a unified states regardless of which store the user is subscribed, we need to first define the “Subscription States”.
We will assign an integer value to each state. The values above 0 will have access to the premium services while 0 and below, will not.
4 ACTIVE
3 TRIAL
2 GRACE
1 CANCEL
0 FREE
-1 SUSPEND
-2 CHURN
-3 REFUNDACTIVE: The user’s bank account has been charged successfully and the subscription is not canceled yet.
TRIAL: The user has subscribed to one of the plans and the subscription is still in the trial period.
GRACE: The store has tried to charge the user’s bank account but it has failed. So the user is currently in the grace period.
CANCEL: The user has canceled the subscription, but the subscription has not been expired yet.
FREE: The user never purchased any subscription.
SUSPEND: The store has failed to charge the user’s bank account and the grace period has also expired.
CHURN: The user has canceled the subscription and the subscription has expired. Or the store has failed to charge the user’s bank account after trying a few times.
REFUND: The user has requested for a refund, directly from the store and successfully refunded.
This is what we are going to implement in the next steps. It’s important to understand that there’s not a single correct answer to this, and every organization should come up with their unique states.
Receipt Digestion
Now, what we need to do is to read the data from the store’s receipt and decide which state the user is in. There are a bunch of different fields in each of the receipts that determines the entitlement state.
Let’s start with the AppStore’s receipts. When you send a request to /verifyReceipt, you receive a JSON object called responseBody (Full documentation here). The responseBody contains many different fields, but the fields that are interesting to us are:
{
... "latest_receipt_info": array,
"pending_renewal_info": array ...
}
- latest_receipt_info: An array that contains all in-app purchase transactions.
- pending_renewal_info: In the JSON file, an array where each element contains the pending renewal information for each auto-renewable subscription.
The latest_receipt_info is an array. To get the latest receipt, you need to pick the item with the most recent purchase_date_ms field. Now within the latest receipt:
{
... product_id: string,
expire_date_ms: string,
is_trial_period: string, ...
}
- product_id: The unique identifier of the product purchased. We will use this to find the related pending_renewal_info.
- expire_date_ms: The time a subscription expires or when it will renew, in UNIX epoch time format, in milliseconds.
- is_trial_period: An indicator of whether an auto-renewable subscription is in the free trial period. The value is always “true” or “false”
After finding the latest receipt from the latest_receipt_info array based on the purchase date field, get the product_id and find the related renewal info from the pending_renewal_info array.
{
... auto_renew_status: string,
is_in_billing_retry_period: string,
grace_period_expires_date_ms: string ...
}
- auto_renew_status: The renewal status for the auto-renewable subscription.
- is_in_billing_retry_period: A flag that indicates Apple is attempting to renew an expired subscription automatically.
- grace_period_expires_date_ms: The time at which the grace period for subscription renewals expires, in UNIX epoch time format, in milliseconds.
With the above information, we can determine the status using this flowchart:
With GooglePlay, you request Purchases:subscriptions endpoint and it responds with a JSON object which the following fields are our interest:
- expiryDateMillis: Time at which the subscription will expire, in milliseconds since the Epoch.
- autoRenewing: Whether the subscription will automatically be renewed when it reaches its current expiry time.
- paymentState: The payment state of the subscription. Possible values are: 0. Payment pending 1. Payment received 2. Free trial 3. Pending deferred upgrade/downgrade.
With the above information in hand, we can use this flowchart to determine the user state:
More information here
4. Store Notifications
We know that in case of a subscription state change, both AppStore and GooglePlay will send us a notification.
This is what you expect to receive from AppStore:
{
"notification_type": string,
"responseBody": object
}- notification_type: The type that describes the in-app purchase event for which the App Store sent the notification.
- responseBody: The JSON data sent in the server notification from the App Store.
The notification_type will tell us what kind of change has happened to the subscription status. You can about the different values and how to interpret them here.
The responseBody contains roughly the same information as the response you would receive from calling the /verifyReceipt endpoint. The main difference is that, all information you need is now under the unified_receipt field.
In GooglePlay’s case however, no subscription info is included. What we receive from the pub/sub channel is a base64 encoded string which after decode will give you the following fields:
{
"version": string,
"packageName": string,
"eventTimeMillis": long,
"oneTimeProductNotification": OneTimeProductNotification,
"subscriptionNotification": SubscriptionNotification,
"testNotification": TestNotification
}The most important field here is subscriptionNotification which contains the following information:
{
"version": string
"notificationType": int
"purchaseToken": string
"subscriptionId": string
}Then we can use packageName, subscriptionId and purchaseToken to query the Purchases:subscriptions endpoint to receive the latest subscription information about the customer.
More details here.
Whenever we receive a notification, we should try to verify it first. After the verification, we can digest the response and change the user’s subscription status accordingly.
Our single source of truth is the verified receipt (AppStore) or the subscription data (GooglePlay). The user’s entitlement state can be completely derived from this. That’s why we are going to ignore the notification types.
Summary
In this part we have reviewed how each of the stores deals with the subscription process. There are both similarities and differences between the two and it is our job to abstract over them and create a unified experience for our users.
AppStore relies on encrypted receipts while GooglePlay returns a token to be kept on our database and used as a key to retrieve the subscription data later. We reviewed flowcharts that show how we can combine multiple fields from the receipts or the subscription data to determine the user’s subscription status.
Both of the stores send notifications whenever a change happens in the user’s subscription status but it’s important not to solely rely on the data received by the notifications and verify that data in the backend right after receiving them.
In the next section, we will write the backend code using Node/Firestore on GCP. See you there!
References
- Architecting for subscriptions.
https://developer.apple.com/videos/play/wwdc2020/10671/ - In-App-Purchases and using S2S notifications.
https://developer.apple.com/videos/play/wwdc2019/302 - Engineering Subscriptions
https://developer.apple.com/videos/play/wwdc2018/705 - AppStore Receipts
https://developer.apple.com/documentation/appstorereceipts - AppStore Server Notifications
https://developer.apple.com/documentation/appstoreservernotifications - Android Documentation: Selling Subscriptions
https://developer.android.com/google/play/billing/subs - Google Play Developer API: Purchases:subscriptions
https://developers.google.com/android-publisher/api-ref/rest/v3/purchases.subscriptions
