Introduction
Most reports and analysis start with two questions about the users:
- How did visitors land on the website?
- Where did visitors land on the website?
How and Where a visitor lands on your store decides what she will do in the next few minutes.
Additionally, we also want to know if a visitor is visiting the store for the first time, or if she is a ‘returning’ visitor. As we will see in this post, we cannot rely on this classification but we can still use it to get some insights.
These details form the identity of a visitor. We treat these details as user attributes – because this is the best information we have about any visitor in the privacy era. For instance, here is a short list of visitors to a t-shirt store:
- new , from Instagram campaign id xyz_1, landed on black t shirt PDP
- new, from organic search, landed on t-shirts collection
- returning, direct visit, landed on home page
- returning, from YouTube campaign xyz_2, landed on white t shirt product display page
- returning, from email, landed on checkout
- new, from email, landed on checkout.
Since this information is used almost everywhere, we will start out data modeling journey with the two events that contain all these details (and more) – first_visit and session_start.
The misunderstood events
Many shopify founders are under the false impression that users have to be converted in every session possible. CRO experts and marketing agencies have created this impression mainly to serve their own goals.
But who makes a purchase from a new brand in the first visit? Very few buyers are that impulsive. Visitors typically make multiple visits to a store before making a purchase – in addition to checking socials, asking on reddit, etc. This is why it’s important to separate first time visitors from returning visitors.
But, due to reasons explained later in this post, we cannot track most of our returning visitors. Most of them will get tagged as new in every visit.
This is the biggest issue with data setups in the privacy era, and there’s very little we can do at the data collection stage.
However, this is where you get to differentiate yourself from most analysts. If you understand how this data is collected by GA4 – you will be able to handle stakeholder questions about new/returning with ease. You will also get better at finding insights from the data.
The majority of this post is dedicated to build your understanding of data collection in the privacy era. This will make all the difference in your data setup.
We will start with a quick look at events and then get into the data collection process, issues in privacy era, and what we can do to handle these issues
Events
Event: first_visit
Trigger: when a new user lands on the website.
The first_visit event is triggered on a user’s first visit to the store. But who decides if it is a first visit – is it shopify, your store, or GA4? Since, all these events are created by GA4 based on what it knows / does not know about your store – the decision is also taken by GA4.
In our data models, we use first_visit event to identify if this is a new user (as per GA4), and get the following details:
- user_first_touch_timestamp: this is the timestamp of the first_visit event. This field is available in all GA4 events.
- user_pseudo_id: this is a unique id given by GA4 to the user (actually to the user’s browser where cookie is stored). This is also available in all events.
- traffic_source: this is the source of user’s first visit, available in all events.
- event_params.page_location: this is the page location where user landed in the first visit.
- ga_session_id: this is the session_id of the first visit.
The event is available for > 99% of visitors. For the other 1%, we have to rely on other events and use ‘user_first_touch_timestamp’ to determine if it’s a new or old user.
Event: session_start
Trigger: Triggered when visitors starts a new session on the store, also triggered on the first session of a visitor.
But what is a session?
A ‘session’ tags continuous user activity on a website with one unique identifier. Each session on a website is a unique journey of 0-10 minutes (can be more) which started on one page and ended on another. All the metrics we calculate are just ways of summarizing sessions.
But a session must have a beginning and an end. Typically, a session ends if user is inactive for some time (30 minutes). A session also ends if user closes the browser.
Once a session ends, user’s next visit creates a new session.
According to GA4 documentation –
- If there is a gap of 30 minutes between user activity, GA4 marks it as a new session. So a visitor landing on your website from 2 Instagram ads will have same session id if the second click is within 30 minutes of the first and different session_id if more than 30 minutes have passed
- if user closes your website’s tabs or the browser, then the next visit gets tagged as a new session – even if it’s after a 1 minute gap.
- if user is on your store on mobile and desktop at the same time – this will be counted as two sessions.
We always analyze user behavior by first creating a session level picture and then aggregating up from there. This is why understanding sessions is critical for analysts.
Key fields from the event:
- ga_session_id
- session_source
- page_location
Data Collection
Gone are the days when you could ignore data collection. To work effectively with website data, you must understand how GA4 collects data.
How does GA4 work?
Cookie time. You must have seen those irritating consent popups when you visit any EU/US based website. Those are asking for your permission to store cookie on your device. Don’t be fooled by the name though, this cookie is just a text file that a website can maintain on your device and update from time to time.
When you visit the site again, the site can lookup the cookie it stored, and read whatever it had stored about you. Then it can customize the website, log you in automatically, or do nothing.
GA4 relies on cookies to tag a user as new or returning. When a new user lands on the store, Google Analytics will store a cookie on the browser – if it is allowed to. When the user visits again, GA4 will find the cookie on the browser and tag the user as returning. It will NOT send the first_visit event.
This is the root of problems in the privacy era. This, and the fact that people don’t really log in to shopify stores. So we can’t rely on login details either except for buyers – something we will deal with in the last post of this series.
Issues with cookies
Cookies worked very well till the early 2010s – before mobile phones replaced desktops for casual browsing, and before instagram and youtube became primary channels of marketing.
Now people don’t like to be tracked anymore (who can blame them?), so they have become bitter to cookies:
- ad blockers, privacy-first browsers and privacy settings on the popular browsers block GA4 and 3rd party services from saving a cookie.
- Cookies are not shared between browsers on a device, or same browser on mobile and desktop. https://www.youtube.com/watch?v=0nGu1sHZDt4
- the cookie on mobile browser (say chrome) is not available to the browser that opens instagram ads (android webview).
- Reinstall or clearing data from the app (instagram) or the browser can delete cookies
- Responding ‘Reject All’ to consent popup prevents GA4 from adding a cookie.
- and then there’s GDPR (for EU based stores) – with those consent popups. If users reject tracking, cookies won’t be setup on their devices
In all these instances, GA4 tags the visitor as ‘New’! Now, do you understand why the ‘New’ users are almost the same as total users? Because only a small fraction can be tracked as returning.
In such cases, GA4 has no way of knowing whether a user is visiting your store for the first or tenth time. GA4 keeps triggering the first_visit event.
It’s a mess.
In most setups, 90% or more users are tagged as New. However, what % are actually new is impossible to find out. The percentage of ‘New’ is significantly lower for older brands with high volume of returning customers.
Impact of this issue on numbers:
- New and Returning user counts are unreliable
- New User numbers are highly exaggerated: Same user on different devices, mobile and desktop, ad channels ( instagram ads and youtube ads) will appear as 4 users. This means your ‘new users’ count could be much smaller than what’s showing up on the dashboard.
- Sessions to User ratio is generally around 1 – implying that no one returns to the store.
Workarounds
Move to session analytics
In the past, when user numbers were reliable, we could trace a user across multiple sessions and understand their journeys. That’s what we did (or tried to do). The unit of analysis was user.
We can’t rely on user anymore. So, we have moved the unit of analysis to sessions.
A session is now the most complete unit of user behavior available to us. To understand user behavior on the store, we have to get what we can from sessions. In this series (and on AhaInsights in general), we focus on session level analysis and reports.
How to share numbers:
- Avoid sharing ‘exact’ numbers of new and returning users with your stakeholders. Always mark them as approximate.
- If you are analyzing user behaviour on a channel-device combination (instagram + mobile) – the numbers are more reliable.
- While the number of returning user is underestimated, the users tagged as returning are correctly tagged – this is a useful cohort for analysis.
- You Can use these as filters in analysis reports -> sure your numbers are not exact, but new users still have majority of ‘new’ and returning users are all returning users.
Data Models
Data Model: Sessions Details
Model: session_details
Description: Contains details of each session. Note that GA4 reports multiple utm source values for each session – traffic_source, collected source, source, etc.. We are interested in ‘source’ – which is the source corresponding to the session.
| Column Name | Description |
|---|---|
| ga_session_id | unique session identifier for the user |
| user_id | unique identifier of the user |
| session_start_time | Timestamp of the first event with this ga_session_id |
| session_end_time | Timestamp of the last event with this ga_session_id |
| session_source | utm source for the session. |
| session_medium | utm medium for the session |
| session_campaign | utm campaign for the session. |
| session_campaign_id | utm campaign id for the session. |
| session_duration | total duration of the session, in seconds. |
| landing_page_og | The URL of the page where the session started. Taken from the page_location of the first event in the session. |
| landing_page | Cleaned URL of the landing page. We remove everything after ‘?’ except variant id. |
| engaged_session_flag | A boolean field that indicates if it was an engaged or non-engaged session (bounce). This is taken from GA4’s engaged_session event param. |
| session_with_key_event | A boolean field that indicates if user performed a key event – add to cart, checkout initiation or purchase – during the session. |
| device_category | Device category of the user (desktop / mobile / tablet) |
| geo_country | Country from where user has accessed the site |
| geo_region | Region from where user has accessed the site |
| geo_city | City from where user has accessed the site |
Data Model: User Details
As explained above, user is not a very reliable entity in data now. Nonetheless, keeping a user table helps us conduct some analysis on the returning users.
Model: user_details
Description: The master table of users who visited the website.
| Column Name | Description |
|---|---|
| user_id | Unique user identifier, taken from user_pseudo_id field in GA4 export. |
| first_session_id | Identifier of the first session of the user on the website. |
| first_visit_ts | Timestamp of the first_visit event of the user, or user_first_touch_timestamp from GA4 export if first_visit event is missing. |
| traffic_source | utm_source for user’s first website visit, from ‘traffic_source’ field in GA4 report. |
| traffic_medium | utm_medium for user’s first website visit |
| traffic_campaign_name | utm_campaign for user’s first website visit |
| traffic_campaign_id | utm_campaign_id for user’s first website visit |
| device_category | User’s device category |
| device_brand_name | User’s device brand name |
| device_operating_system | User’s device OS |
| geo_country | User’s country – from first visit event. |
| geo_region | User’s region – from first visit event. |
| geo_city | User’s city – from first visit event. |
| sessions | Number of sessions by the user on the site |
| last_active_ts | Timestamp, in micros, when last session ended. |
FAQs
No, the event is not always available. It is not available for a very small percentage of data (<1%). But ga_session_id if present in all events as a event_parameter even if the session_start event is missing. ga_session_id is the timestamp of the session start time in seconds.
So, when building the data model, we don’t filter for session_start event. We take all events and we get all the sessions from the event parameters ga_session_id parameter.
GA4 defines session source as last non-direct click in the past 90 days. So if a user comes to your website through a referral, marketing channel or an organic search, subsequent direct visits will be tagged to the previous channel.
This is in line with how Shopify reports orders and revenue on it’s dashboards. Since marketing platforms like Meta Ads take credit for a purchase in a 30 day window, this keeps the data consistent across platforms.
However, you can use the utm_source field page_referrer event parameter to get the source of any session.
According to GA4 documentation, engaged session is one with more than 10 seconds on the platform, or 2 clicks, or 1 key event. By default, only ‘Purchase’ is considered as a key event but you can add more from the GA4 console.
Leave a Reply