Most identity pilots fail quietly. They start with enthusiasm, drift into a proof of concept that never ends, and finish with a vague feeling that it "seemed fine".
Thirty days is long enough to learn what matters and short enough to keep attention. This plan is vendor-neutral: use it for any platform you are evaluating, including a replacement for what you have.
Before day one: decide what you are testing
A pilot answers specific questions. Write them down first. Good ones look like this:
- Can people sign in to our ten most-used applications through one login?
- Does joiner and leaver provisioning work from our HR source of truth?
- Can administrators do their daily tasks without calling the vendor?
- Will staff accept the second factor we want to require?
- Can we get the logs we need into our monitoring?
Pick no more than five. Every extra question dilutes the result.
Choose the scope
- Applications: six to ten, mixing a few high-use ones, one or two awkward ones (older, odd protocol), and at least one that needs automated provisioning. Avoid the single most business-critical system in the first round.
- People: 30 to 60 users across at least three departments, including IT, someone from HR operations, a manager, and a few sceptics. Pilots made only of enthusiasts produce flattering results.
- Environment: production identities with real applications, if you can. A sandbox hides the problems you care about.
Assemble the group
| Role | Responsibility |
|---|---|
| Sponsor | Removes blockers, owns the final decision |
| Pilot lead | Runs the plan, keeps the weekly log |
| IT administrator | Configures, tests, handles tickets |
| HR operations | Supplies joiner and leaver data, validates results |
| Security | Reviews logs, factor policy, recovery process |
| Compliance or legal | Reviews data location and contract terms |
| Two or three end users | Honest feedback, early |
Tip
Write the success measures in week one. A pilot without numbers is a demo.
Week 1: Foundation
Goal: one person can sign in to one application, and you can see what happened.
- Set up the tenant, administrator roles and an emergency access account stored offline.
- Connect your user source. Import the pilot group only, not the whole company.
- Integrate the first two applications: one using SAML 2.0 and one using OpenID Connect, if your estate has both. (SSO basics explains the difference.)
- Turn on audit logging and confirm you can export it.
- Record the baseline: how long a new-starter setup and a leaver cleanup take today, and how many password-reset tickets you receive per week.
Exit check for the week: the pilot lead can demonstrate a sign-in end to end and show the matching log entries.
Week 2: Applications and factors
Goal: the application scope is complete and the second factor is live for the pilot group.
- Integrate the remaining pilot applications. Keep a table of what worked, how long it took, and any workaround required.
- Enrol the pilot group in the second factor you plan to require. Run a short briefing, not a long training session.
- Test the recovery path deliberately: someone loses a phone, someone changes devices.
- Test administrator actions that need stronger checks, such as step-up authentication.
- Start a support log. Every question or failure goes in, with time to resolve.
Exit check: at least 90 percent of the pilot group has enrolled a factor, and every pilot application has at least one successful sign-in from each department.
Week 3: Lifecycle and logs
Goal: prove that people arriving, moving and leaving are handled without manual steps. This is where platforms differ most. Read the joiner-mover-leaver checklist and replay it.
- Create a real or test joiner in the HR source and time how long until accounts appear in the pilot applications.
- Change a department and check that access changes with it, and that old access is removed rather than only new access added.
- Deactivate a test leaver and verify sessions end and accounts are disabled everywhere in scope, not only in the platform.
- Send audit events to your monitoring tool and write one detection. See streaming identity logs to your SIEM.
- Ask your compliance contact to review where data is stored and processed, and the supplier terms.
Exit check: a joiner, a mover and a leaver have each completed correctly, with timestamps recorded.
Week 4: Stress, measure, decide
Goal: gather the evidence and make the decision.
- Run a failure exercise: block the platform from a test network and see how people and administrators cope. Confirm the emergency account works.
- Repeat the baseline measurements and compare them.
- Survey the pilot group with five questions, no more. Include one free-text field.
- Hold a review with the whole project group, bring the support log and the metrics, and apply the exit criteria below.
What to measure
Choose numbers you can collect without effort:
- Sign-in success rate on first attempt.
- Median time to sign in, compared with before.
- Password-reset and lockout tickets per week.
- Time from HR change to access change, for joiners and leavers.
- Number of applications integrated without a workaround.
- Enrolment completion rate for the second factor.
- Support tickets raised per user, and median time to resolve.
- Number of manual steps still needed per joiner.
Avoid vanity measures such as "number of features tested". They tell you nothing about operating the system.
Success and exit criteria
Agree these in week one, in writing, and do not adjust them afterwards:
- At least 80 percent of pilot users rate sign-in "easier or same" than before.
- No unresolved critical issue in the support log.
- Joiner and leaver tests pass with no manual workaround.
- Audit logs reach your monitoring tool and answer a "who did what, when" question in under ten minutes.
- Administrators complete defined tasks without vendor support.
- Security and compliance reviewers sign off in writing, or list the specific conditions.
The decision then has three outcomes: proceed to rollout, extend the pilot by two weeks with named gaps, or stop. Stopping is a valid, useful result.
Plan the rollback before you need it
A pilot without a rollback is a commitment in disguise. For each application, record:
- The original sign-in method and who knows how to restore it.
- Whether local administrator accounts still exist in the application. Do not remove them during the pilot.
- A tested way to switch the pilot group back within one working day.
- What happens to enrolled factors and data if you leave, including how to export your own records.
Practise the rollback once on a single application during week four.
What to do this week
- Write your five questions and your six exit criteria on one page.
- Name the sponsor and the pilot lead.
- Pick ten applications and mark the awkward ones.
- Collect your baseline numbers now, before anything changes.
- Book the week-four review in everyone's calendar today.