How to get source and medium programmatically with JavaScript - just like Google Analytics

-

To make smart marketing decisions, you need to know what the Return On Marketing Investment (ROMI) is. When you're a webshop that does immediate transactions, it's easier to set up Google Analytics to serve you. However, if you need to understand longer customer relationships, Lifetime Values, etc, you need to get your hands on raw data.

The obvious idea that comes to mind is: "I just need to get the source, medium, campaign etc. data per each visitor" and you quickly realize that Google Analytics doesn't allow you to do that.

You need a custom solution. I have found two online that solve the problem

  1. This one with FirstSession and ReturningSession cookies set
  2. The Lunametrics one, which is a bit more verbose, but has a more extensive list of search engines

These solutions are just fine, but, no-one's maintaining them. They're old. They don't evolve. And they don't support other paid channels than Google search. What about Bing? Facebook?

Also, both of these solutions combine two very separate problems into one:

  1. Deciphering what the traffic data is (source, medium, campaign etc), and
  2. Storing it for later retrieval.

And the 2nd part happens through cookies. It's 2022 and Cookies could soon be dead. Then what?

VisitDataJs: the open source Google Analytics data emulator

I've written a small javascript open source library to tackle the first problem - getting Google Analytics data such as source and medium, programmatically through JavaScript - called VisitDataJS.

Usage is simple and you should refer to the readme, but simply include the javascript and run it, like so:

<script src="https://cdn.jsdelivr.net/npm/visitdata/dist/visitdata.umd.js"></script>
<script>
	console.log( visitData.get() );
</script>

Here's the results for you personally, how you arrived to this page:

GlobalStorage: the open source, cross domain browser storage that doesn't run on cookies

GlobalStorage is an open source javascript library you can include from anywhere, and use it just like you use the browser's localStorage, except that you can access the same data from any website! Here's how it works:

<script src="https://cdn.ilkkapeltola.com/global-storage/latest/globalStorage.js"></script>
<script>
  globalStorage.init().then(() => {
    globalStorage.setItem("your-key-name", { foo: "bar" } );
  });
</script>

The above code will store the object { foo: "bar" } to the key 'your-key-name', which can then be fetched from another web page by calling getItem. You should check the readme for the library as there are some default behavior of this you need to be aware of, such as how do you control who has the right to read the data.

Years ago already, Zendesk did a marvelous job with Cross Storage, but there was one thing I think they got wrong: forcing you to host your own 'Hub' file to have control over who has access to your data. So I took a lot from Zendesk's cross storage but made a completely new version that achieves something similar, but more easily. While Zendesk's Cross Storage requires you to also host a hub.html file and configure it for yourself, my solution works easily without the need for you to host your own file. You can just use the one I hosted - but you still have control over your data.

Incredible value having access to this data

We've gained an incredible amount of value by having access to this data. It's not about being able to replicate GA:s reports, but instead, enriching our user base with new dimensions that we can use as a lens into business metrics. For example:

  • Are users acquired by paid acquisition more likely to churn than those who we acquire by organic means? 
  • Which campaigns are most likely to convert people into active and paying customers?
  • What is the lifetime value of a new conversion, split by cpc vs organic?
What we've found out for example is that CPC traffic can convert, often better than organic, but the bottom of the funnel often performs worse. We've been able to identify Google Ads campaigns that bring in signups, but never bring paying customers, and cancel them. We've seen how affiliate networks we've tried have lead to fraudulent accounts and no business. These things are difficult to prove without granular, per account data.

Comments

Post a Comment

Popular posts from this blog

How to access AWS S3 with pyspark locally using AWS profiles tutorial

-
At Zervant , we currently use databricks for our ETL processes, and it's quite great. However, there's been some difficulty in setting up scripts that work both locally and on the databricks cloud. Specifically, databricks uses their own prorpietary libraries to connect to AWS S3 based on AWS hadoop 2.7. That version does not support accessing using AWS profiles. Internally, we use SSO to create temporary credentials for an AWS profile that then assumes a role. Therefore, reading the ACCESS_ID and ACCESS_SECRET from the .credentials file is something we don't want to do. In order to accomplish this, we need to set two hadoop configurations to the Spark Context fs.s3a.aws.credentials.provider com.amazonaws.auth.profile.ProfileCredentialsProvider This is done by running this line of code: sc._jsc.hadoopConfiguration().set("fs.s3a.aws.credentials.provider", "com.amazonaws.auth.profile.ProfileCredentialsProvider") Note! You need to set your environment var...
Read more

Snowflake UPSERT operation (aka MERGE)

-
You want to insert data to a table, but if a corresponding row already exists (by some rule, e.g. unique key) you want to update that instead of adding a new row, keeping the dataset's unique requirements intact. That's an "UPDATE AND INSERT" operation, or UPSERT. Some SQL languages have native support for it.  PostgreSQL has UPSERT as native. Also MySQL supports the operation with INSERT and ON DUPLICATE KEY UPDATE. How do you do UPSERT on Snowflake? Here's how: Snowflake UPSERT i.e. MERGE operation Snowflake's UPSERT is called MERGE and it works just as conveniently. It just has a different name. Here's the simple usage: MERGE INTO workspace.destination_table d USING workspace.source_table s ON d.id = s.id AND d.val1 = s.val1 WHEN MATCHED THEN update SET d.val2 = s.val2, d.val3 = s.val3 WHEN NOT MATCHED THEN INSERT (id, val1, val2, val3) VALUES (s.id, s.val1, s.val2, s.val3); Here the destination_table and source_table are of similar form,...
Read more