FORUM Product Analytics

Product Analytics

Let’s discuss how data helps us build better products. Get support and learn about product metrics like AARRR or Unit Economics, how to analyze engagement and retention, how to make data help us take quick and great decisions 🚀

When to categorize or assign users in A/B testing?

AB-test categorization. Part 1 - When categorization happens

When it’s mentioned that “A great way to categorize users is during the signup process”, is this specifically for testing onboarding flow changes or all changes?

The lesson also cautions against “categorizing users when they visit this page or screen” that’s subject to AB test changes. But why? Wouldn’t you want to assign users to their A/B variant at the point of change as not all users will go through the funnel and you risk requiring a larger sample size? The concern I’m flagging is also outlined here: https://medium.com/@foundinblank/why-it-matters-where-you-randomize-users-in-a-b-experiments-5570c7585944.