How I Changed the Way a 100-Person Marketing Team Made Decisions
What it actually takes to move an organization from opinion to evidence, and why the hardest part was never the data.
This article covers what it actually takes to move a 100-person marketing organization from opinion-based to evidence-based decisions, including the behavioral data infrastructure required, the experimentation program design, and the specific shift in how stakeholders began asking for work.
I was hired to improve conversion optimization and testing at an enterprise life sciences platform. The team already had a strong analytical culture, solid traffic analysis, and the academic rigor you expect from people who argue from evidence. What they did not have was the behavioral layer. They could see how many people arrived and where from. They could not see how people actually behaved on the page or settle which design or message worked, except by debating it.
That was the gap I was brought in to close. And the most useful thing the work did was not raise a number. It ended arguments. Debates that used to be won by whoever argued best started getting settled by behavioral data instead.
Three years later, the change I am most proud of is not a single test result. It is that the organization learned to decide by evidence in the places where it used to decide by debate. Here is how that happened and what I would tell anyone trying to do the same.
The foundation came first
You cannot settle an argument with data you do not trust, and for on-page behavior there was no data to trust yet. So the first work was not glamorous. I established a single behavioral data source, defined and documented every conversion event against the actual sales journey, and built naming conventions so that two people looking at the same report saw the same thing. I built dashboards in Data Studio, formerly named Looker Studio, in formats a non-technical stakeholder could read without a translator.
Then I added behavioral analytics. I incorporated Lucky Orange heatmapping, which gave the team a simple, shared definition of success that anyone in the room could understand: the effective fold, the point on a page where the data showed half of the traffic had dropped off. Instead of debating whether something was above the fold in theory, we could see where attention actually ended and designed to move it further down the page.
A shared definition of success ends more arguments than a better argument does.
I made testing a process, not a favor
Once the behavioral data was trustworthy, I built the organization’s first structured experimentation program. Not a one-off test. A repeatable system:
- A hypothesis intake process, so any team could submit a test idea in a consistent format
- An ICE prioritization framework, which ranks ideas by Impact, Confidence, and Ease, so we chose tests by expected value instead of by who suggested them
- A shared results repository, so every outcome was visible to everyone, including the tests that failed
- A minimum testing rhythm each quarter, so experimentation became a habit rather than an event
I focused the program on two things at once: tests that improved conversion and tests that established best practices for the website. Many of those tests did something the team had never been able to do before. They settled long-running internal debates with actual data instead of opinion.
I want to be clear about how this worked because it is the part that scales. I did not run the tests alone. For each one, I designed the experiment, calculated the reach and the runtime it needed to produce a trustworthy result, and decided how it would be analyzed. Then I worked with developers, designers, and copywriters to build and launch it, coordinating the effort and stopping any test the moment the setup or the data was not sound. My role was to own the experiment and direct the people who brought it to life. Experimentation at real scale is a team effort, and running it well is a leadership task, not a solo one.
I wrote a manual, not a rulebook
Most organizations adopt a generic set of best-practice rules from a blog post or a vendor. I did the opposite. I designed a customized best-practices manual built specifically for the business. It paired a business-language reference, the approved vocabulary that kept teams aligned, with behavioral patterns I had defined through our own testing and analytics. The rules were ours, proven on our own audience, written in the language our stakeholders already used.
The hardest part was the people, and the way I spoke to them
The technical work was the easy half. Experimentation threatens people because a test that can prove an idea wrong can prove a stakeholder wrong, and in a cross-functional environment with territorial ownership, that is a political problem before it is an analytical one.
What worked was framing, not force. I reframed every result in terms of business outcomes rather than metric movements, used collaborative language, and treated stakeholders as partners whose ideas were being tested, not judged. I also changed how I presented. I built and delivered every presentation the way a decision maker would present to the CEO and the board: not an analyst walking through charts, but someone leading with the business decision the data supported. Over time, the people who were most resistant became the ones requesting tests, because the process made them look good when they were right and protected them when they were wrong.
Culture change in analytics is not a data problem. It is a trust problem wearing a data costume.
What actually changed
By the end, the same organization that settled design and messaging questions by debate was submitting hypotheses, waiting for results, and citing evidence in rooms where it used to cite seniority. One full-page restructure, grounded in behavioral data and run as a structured test, produced a 40 percent increase in traffic to a key program. But the restructure is not the point. The point is that nobody had to be convinced to test it, because testing was simply how decisions were made by then.
It changed how people asked for work, too, from ‘how do we get more clicks?’ to ‘I need to increase this metric; how do we do that?’
If you are the person trying to make an organization data-driven, here is the short version. Build the foundation before you ask anyone to trust it. Make testing a process with rules, not a series of favors. And spend most of your energy on the people, because the analytics were never the thing standing in your way.
There is a larger version too. The way you protect business growth through a difficult economy is the same way you protect it in a good one. You do not guess harder. You build on foundations you actually own, you build real relationships with the people who make the decisions, and you follow a path the data can defend. I have relied on that pattern through more than one hard market, including keeping a company’s primary revenue channel alive through the pandemic. In a strong economy, it gives you an edge. In a weak one, it is the only thing that works.
What finally moved your organization from opinion to evidence?
This article is a summary of Case Study 1, available in full inside the Work section upon request.
The problem was not data. It was trust.
When I joined an enterprise life sciences platform as its first dedicated CRO hire, the marketing team already had strong analytical instincts. People argued from evidence. What they lacked was the behavioral layer: no data on how people actually behaved on pages, no agreed method for settling design and messaging decisions, and no structured way to test ideas. Debates were won by whoever argued best, not by whoever had data.
Foundation before process
The first thing I built was not a test. It was a measurement infrastructure the organization could trust. I migrated all conversion tracking to GA4, defined every conversion event against the actual sales journey, and implemented Lucky Orange behavioral analytics across key landing pages. I introduced a concept I call the effective fold: the point on any page where heatmap data showed half the traffic had dropped off. That single reference point ended more debates than any chart I could show, because it gave everyone in the room the same definition of where a page’s attention ended.
Process before culture
Once the data was trustworthy, I built a repeatable testing process. A hypothesis intake form so any team could submit an idea in a consistent format. An ICE/PIE prioritization framework so decisions were made by expected value, not by stakeholder seniority. A shared results repository, including failed tests, so the whole organization could see what the data showed. A minimum testing rhythm each quarter so experimentation became a scheduled activity rather than an ad-hoc favor.
The shift
The culture shift showed up in how people asked for work. Requests moved from how do we get more clicks to I need to increase this number, how do we do that. That change in language was the clearest evidence that the shift had taken hold. The process enables the culture, but the culture is the goal.
What this means in practice
Building a CRO program is not a data problem. It is a trust problem that needs data to solve it. The technical work is achievable in the first few months. The shift that matters, when a senior stakeholder submits a hypothesis and waits for results instead of calling a meeting, takes sustained trust-building across the whole organization.
The full case study, including the specific tests run, the frameworks used, and the measurable outcomes, is available upon request.
Request Access to the Full Case Study