Posts

Showing posts from 2018

Using survival plot to analyze churn in Power BI

-
Image
I did not  guess I'd be working with Kaplan-Meier survival plots so soon. Analyzing churn and trying to figure out what kind of users churn more likely is not so easy. In order to calculate churn, you need to have a good volume of users that you can follow month over month, having a number of active users per month and what share of them are left behind every month. But what if you want to select a different set of users, another segment? Producing an analytics cube with the necessary dimensions takes time. And if you end up with a segment that doesn't have high volumes every month, interpreting the results can be quite tricky.
Post a Comment
Read more

Create a funnel analysis tool with Redshift and Power BI in 5 minutes

-
Image
If you're not collecting events from your product, get started right away! Events are a great way to collect behavioral data on how your users use your data: what paths they take, what errors they encounter, how long something takes etc. When you have events, there isn't a lot you cannot  analyze.
Post a Comment
Read more

How to ETL in Amazon AWS? AWS Glue for dummies

-
Image
You can do ETL in AWS in a few different ways: Glue DataPipeline A custom solution, e.g. a Docker
Post a Comment
Read more

An insights strategy for winning companies

-
Image
An executive summary Companies struggle to gain maximum benefit from analytics and insights since Analytics is seen as a support function, not a business partner and therefore is  not prioritized high enough Analytics is separated from business processes and insights are produced away from execution Under-resourced, inflexible analytics stack, which doesn’t enable speed to react to changing needs
Post a Comment
Read more

A more holistic purpose of analytics

-
Image
Why do people make bad decisions? Mostly because of insufficient or wrong information, but not always. So what should be done about it?
Post a Comment
Read more

Simple Big Data setup on Amazon AWS

-
Image
Everyone wants to do some big data stuff, right? In all honesty, no-one cares if your data is big or small - size doesn't matter. What matters is your ability to take any size of data and generate understanding from it. At some point the data you are gathering might become inconvenient to process with more traditional tools. It might  be that some big data tools might help you - or not. The bottom line is, it is a tool you want to have in your toolbox.
Post a Comment
Read more

Simple way to query Amazon Athena in python with boto3

-
Image
ETL takes time and it's a lot to maintain. Sometimes it breaks when you didn't expect a string to contain emojis. You might decide the transformation needs to be changed, which means you need to refresh all your data. So what can you do to avoid this?
4 comments
Read more

Know exactly how much you pay to acquire any user: Python with Google API

-
Image
So you've read about how to optimize your marketing efforts through data . With that, you should know the kind of users different marketing campaigns are bringing in. Some campaigns might be bringing more high-quality users than others. Can you do that now?
Post a Comment
Read more

What is the easiest way to train a neural network?

-
Image
We've been surveying our customers since forever, but recently we've started to become more and more hungry on data. As an invoicing service provider for micro entrepreneurs, Zervant's  customers are of all varieties. One interesting data snippet we've found out is that roughly half of our customers are part-time entrepreneurs. But which half is it?
Post a Comment
Read more

Prioritize your features with the Kano method

-
Image
Prioritizing new features requires intuition, but it can definitely be made better by using data. As there's always a limited amount of bandwidth to do stuff, you need to relentlessly prioritize what to do next. An easy way to fail is to be too busy doing the wrong things.
Post a Comment
Read more

Popular posts from this blog

How to access AWS S3 with pyspark locally using AWS profiles tutorial

-
At Zervant , we currently use databricks for our ETL processes, and it's quite great. However, there's been some difficulty in setting up scripts that work both locally and on the databricks cloud. Specifically, databricks uses their own prorpietary libraries to connect to AWS S3 based on AWS hadoop 2.7. That version does not support accessing using AWS profiles. Internally, we use SSO to create temporary credentials for an AWS profile that then assumes a role. Therefore, reading the ACCESS_ID and ACCESS_SECRET from the .credentials file is something we don't want to do. In order to accomplish this, we need to set two hadoop configurations to the Spark Context fs.s3a.aws.credentials.provider com.amazonaws.auth.profile.ProfileCredentialsProvider This is done by running this line of code: sc._jsc.hadoopConfiguration().set("fs.s3a.aws.credentials.provider", "com.amazonaws.auth.profile.ProfileCredentialsProvider") Note! You need to set your environment var...
Read more

Snowflake UPSERT operation (aka MERGE)

-
You want to insert data to a table, but if a corresponding row already exists (by some rule, e.g. unique key) you want to update that instead of adding a new row, keeping the dataset's unique requirements intact. That's an "UPDATE AND INSERT" operation, or UPSERT. Some SQL languages have native support for it.  PostgreSQL has UPSERT as native. Also MySQL supports the operation with INSERT and ON DUPLICATE KEY UPDATE. How do you do UPSERT on Snowflake? Here's how: Snowflake UPSERT i.e. MERGE operation Snowflake's UPSERT is called MERGE and it works just as conveniently. It just has a different name. Here's the simple usage: MERGE INTO workspace.destination_table d USING workspace.source_table s ON d.id = s.id AND d.val1 = s.val1 WHEN MATCHED THEN update SET d.val2 = s.val2, d.val3 = s.val3 WHEN NOT MATCHED THEN INSERT (id, val1, val2, val3) VALUES (s.id, s.val1, s.val2, s.val3); Here the destination_table and source_table are of similar form,...
Read more