Migration Planning Guide

Before starting the technical migration process, careful planning is essential to ensure a smooth transition. This guide covers critical planning considerations including merging multiple data sources, defining your user attribute structure, and handling social providers and verification methods.

Prerequisites

Review the Migration Overview to understand different migration approaches, then use this guide to plan your specific migration strategy.

Merging Multiple Data Sources
Defining User Attribute Structure
Handling Social Providers
Handling Verification Methods
Migration Planning Checklist

Merging Multiple Data Sources

When migrating from multiple systems or data sources, you need a clear strategy to merge user data while maintaining data integrity and avoiding duplicates.

Understanding the Challenge

Merging multiple sources presents several challenges:

Duplicate Users: The same user may exist in multiple systems with different identifiers
Conflicting Data: Different systems may have different values for the same attribute
Incomplete Data: Some attributes may only exist in one source system
Data Quality: Varying data quality across different sources

Step 1: Identify User Identity Keys

Determine how to uniquely identify the same user across different sources:

Source System	Primary Identifier	Secondary Identifiers	Notes
System A	`email`	`username`, `customer_id`	Email is verified
System B	`user_id`	`email`, `phone`	Email may be unverified
System C	`account_number`	`email`, `social_id`	Legacy system

Decision Criteria:

Use email as primary identifier if it's verified and unique across systems
Use mobile number if email is not reliable
Create a mapping table if no single identifier is reliable
Consider fuzzy matching for cases where identifiers differ slightly

Step 2: Define Merge Strategy

Choose how to handle conflicts when the same user exists in multiple sources:

Strategy A: Source Priority

Assign priority to each source system. Higher priority sources override lower priority ones.

Example:

Priority 1: CRM System (most up-to-date)
Priority 2: Legacy Auth System (has passwords)
Priority 3: Marketing System (has preferences)

Implementation:

{
  "mergeStrategy": "priority",
  "sources": [
    {"name": "crm", "priority": 1, "fields": ["email", "given_name", "family_name", "customFields"]},
    {"name": "auth", "priority": 2, "fields": ["password_hash_info", "userStatus"]},
    {"name": "marketing", "priority": 3, "fields": ["customFields.newsletter_consent"]}
  ]
}

Strategy B: Most Recent Wins

Use the most recently updated value for each field.

Use when:

All sources are equally reliable
You have reliable timestamp data
You want to preserve the latest information

Strategy C: Field-Specific Rules

Define specific merge rules for each field type.

Example Rules:

Email: Use verified email over unverified
Password: Use the most recent password hash
Profile Data: Use the most complete data set
Custom Fields: Merge arrays, use latest for single values

Step 3: Handle Duplicate Detection

Implement duplicate detection before migration:

Pre-Migration Deduplication

Extract all users from all source systems
Normalize identifiers (lowercase emails, format phone numbers)
Identify potential duplicates using matching algorithms
Review and confirm duplicates manually or with business rules
Create merge mapping for confirmed duplicates

Matching Algorithms

Exact Match:

Same email address
Same mobile number
Same unique identifier

Fuzzy Match:

Similar email addresses (typos, domain variations)
Similar names with same identifier
Same phone number with different formatting

Example Deduplication Logic:

function identifyDuplicates(users) {
  const duplicates = [];
  const emailMap = new Map();
  
  users.forEach(user => {
    const normalizedEmail = user.email.toLowerCase().trim();
    if (emailMap.has(normalizedEmail)) {
      duplicates.push({
        primary: emailMap.get(normalizedEmail),
        duplicate: user,
        matchType: 'email'
      });
    } else {
      emailMap.set(normalizedEmail, user);
    }
  });
  
  return duplicates;
}

Step 4: Create Unified User Structure

Before migrating to cidaas, create a unified user structure that combines data from all sources:

{
  "sub": "generated-or-mapped-sub",
  "email": "[email protected]",
  "email_verified": true,
  "mobile_number": "+49123456789",
  "mobile_number_verified": true,
  "given_name": "John",
  "family_name": "Doe",
  "userStatus": "VERIFIED",
  "provider": "self",
  "password_hash_info": {
    "algorithmTypeId": "BCRYPT_10",
    "passwordHash": "$2b$10$..."
  },
  "identities": [
    {
      "provider": "facebook",
      "providerUserId": "fb_12345",
      "email": "[email protected]"
    },
    {
      "provider": "google",
      "providerUserId": "google_67890",
      "email": "[email protected]"
    }
  ],
  "customFields": {
    "customer_id": "CUST-12345",
    "legacy_system_id": "LEG-67890",
    "newsletter_consent": true,
    "marketing_source": "website"
  }
}

Step 5: Migration Order

When merging multiple sources, migrate in a specific order:

Primary Source First: Migrate the most authoritative source
Secondary Sources: Add identities and additional data
Verification: Ensure all merged users are correctly linked

Example Migration Sequence:

Week 1: Migrate primary auth system (passwords, basic profile)
Week 2: Add social provider identities from social login system
Week 3: Merge custom fields and preferences from CRM
Week 4: Verify and reconcile any conflicts

Defining User Attribute Structure

Before migration, clearly define which user attributes will be stored in cidaas and how they will be structured.

Step 1: Categorize Attributes

Classify attributes into system fields and custom fields:

System Fields (Built-in)

cidaas provides standard OIDC/OAuth2 system fields:

Field	Type	Required	Description
`sub`	String	Yes	Unique user identifier
`email`	String	Recommended	Primary email address
`email_verified`	Boolean	Recommended	Email verification status
`mobile_number`	String	Optional	Mobile phone number
`mobile_number_verified`	Boolean	Optional	Mobile verification status
`given_name`	String	Optional	First name
`family_name`	String	Optional	Last name
`middle_name`	String	Optional	Middle name
`nickname`	String	Optional	Display name
`username`	String	Optional	Username
`userStatus`	Enum	Yes	VERIFIED, PENDING, DECLINED, DELETED
`provider`	String	Yes	Authentication provider (self, facebook, etc.)
`birthdate`	Date	Optional	Date of birth
`locale`	String	Optional	User locale
`picture`	String	Optional	Profile picture URL

Custom Fields

Define custom fields for business-specific data:

Considerations:

Data Type: String, Number, Boolean, Date, Array, Object
Required vs Optional: Which fields are mandatory?
Validation Rules: Format, length, pattern requirements
Sensitivity: PII, financial, health data considerations

Example Custom Fields:

{
  "customFields": {
    "customer_id": "CUST-12345",           // String
    "loyalty_points": 1250,                // Number
    "preferred_language": "en",            // String
    "newsletter_consent": true,            // Boolean
    "registration_date": "2024-01-15",    // Date (as string)
    "preferred_categories": ["electronics", "books"], // Array
    "billing_address": {                   // Object
      "street": "123 Main St",
      "city": "Berlin",
      "zip": "10115"
    }
  }
}

Step 2: Field Mapping Strategy

Create a mapping document from source systems to cidaas:

Source Field	Source System	cidaas Field	Transformation	Notes
`user_email`	System A	`email`	Lowercase, trim	Primary identifier
`email_verified`	System A	`email_verified`	Boolean conversion	Map "yes" → true
`first_name`	System A	`given_name`	Title case	Capitalize first letter
`last_name`	System A	`family_name`	Title case	Capitalize first letter
`phone`	System A	`mobile_number`	E.164 format	Add country code
`status`	System A	`userStatus`	Enum mapping	"active" → "VERIFIED"
`customer_number`	System B	`customFields.customer_id`	Direct	Business identifier
`preferences`	System B	`customFields.preferences`	JSON stringify	Complex object

Step 3: Data Quality and Validation

Define validation rules for each field:

Email Validation:

Format validation (RFC 5322)
Domain validation
Disposable email detection (configurable)
Uniqueness check

Mobile Number Validation:

Format validation (E.164 recommended)
Country code validation
Uniqueness check (if used as identifier)

Custom Field Validation:

Type checking
Format validation (regex patterns)
Length constraints
Value constraints (enums, ranges)

Step 4: Field Setup in cidaas

Before migration, configure fields in cidaas:

Enable Required System Fields
- Navigate to Field Settings
- Enable all system fields you plan to use
- Configure field requirements (required, optional, hidden)
Create Custom Fields
- Define custom field names
- Set data types
- Configure validation rules
- Set field visibility and editability
Test Field Configuration
- Create test users with all field types
- Verify validation rules work correctly
- Test field display in hosted pages

Migrating users with social provider accounts (Facebook, Google, Apple, etc.) requires special consideration to maintain account linking and authentication capabilities.

In cidaas, social providers are represented as identities linked to a user account. A single user can have multiple identities (e.g., email/password + Facebook + Google).

From your source systems, identify users who have social provider accounts:

Questions to Answer:

Which social providers are currently supported?
How many users have social provider accounts?
Do users have multiple social providers linked?
Are social provider accounts the primary authentication method?

Extract and map social provider information:

Source Data	cidaas Identity Field	Example
Provider name	`provider`	"facebook", "google", "apple"
Provider user ID	`providerUserId`	"1234567890"
Provider email	`email`	"[email protected]"
Provider username	`username`	"johndoe"
Provider picture	`picture`	"https://..."

Strategy A: Migrate as Identities

If users have a primary account (email/password) and social providers:

Migrate primary account first with email/password
Add social provider identities to the same user using the identities array
Ensure consistent sub across all identities

Example:

{
  "sub": "8dfc4306-9f3b-4a33-95d0-98c8954da2c1",
  "email": "[email protected]",
  "provider": "self",
  "password_hash_info": {...},
  "identities": [
    {
      "provider": "facebook",
      "providerUserId": "fb_123456789",
      "email": "[email protected]"
    },
    {
      "provider": "google",
      "providerUserId": "google_987654321",
      "email": "[email protected]"
    }
  ]
}

If users only have social provider accounts (no email/password):

Use social provider email as primary identifier
Set provider to the social provider name
No password hash required
Consider email verification status from provider

Example:

{
  "sub": "8dfc4306-9f3b-4a33-95d0-98c8954da2c1",
  "email": "[email protected]",
  "email_verified": true,
  "provider": "google",
  "identities": [
    {
      "provider": "google",
      "providerUserId": "google_987654321",
      "email": "[email protected]"
    }
  ]
}

Step 4: Account Linking Considerations

Prevent Duplicate Accounts:

When migrating social provider users, ensure you don't create duplicate accounts:

Check user existence before creating new account
Use email as identifier to link social provider to existing account
Use sub consistently when adding identities to existing users

Example Linking Logic:

async function migrateSocialUser(socialUser) {
  // Check if user with this email already exists
  const existingUser = await checkUserExistence(socialUser.email);
  
  if (existingUser) {
    // Add social provider identity to existing user
    return await addIdentityToUser(existingUser.sub, {
      provider: socialUser.provider,
      providerUserId: socialUser.providerUserId
    });
  } else {
    // Create new user with social provider as primary
    return await createUserWithSocialProvider(socialUser);
  }
}

Step 5: Post-Migration Verification

After migrating social provider users:

Test social login for each provider
Verify account linking works correctly
Check user profile shows all linked providers
Test account unlinking if supported

Handling Verification Methods

Users may have various verification methods (MFA, email verification, phone verification) that need to be migrated or reconfigured.

Step 1: Identify Verification Methods

Determine which verification methods exist in your source systems:

Verification Method	Source System	Migration Approach
Email verification	System A	Migrate `email_verified` status
Mobile verification	System A	Migrate `mobile_number_verified` status
Password	System A	Migrate `password_hash_info`
TOTP (Authenticator)	System A	Cannot migrate (re-enroll)
Backup codes	System A	Cannot migrate (regenerate)
FIDO2/WebAuthn	System A	Cannot migrate (re-enroll)
Pattern/PIN	System A	Cannot migrate (re-enroll)

Step 2: Migratable vs Non-Migratable Methods

Migratable Verification Methods

These can be migrated directly:

Email Verification:

{
  "email": "[email protected]",
  "email_verified": true  // Migrate status from source
}

Mobile Number Verification:

{
  "mobile_number": "+49123456789",
  "mobile_number_verified": true  // Migrate status from source
}

TOTP (Time-based One-Time Password): If your source system uses TOTP and cidaas supports importing TOTP secrets:

{
  "verificationMethods": [
    {
      "method": "TOTP",
      "friendlyName": "Google Authenticator",
      "secret": "base32_encoded_secret",
      "verified": true
    }
  ]
}

Note: TOTP secret migration depends on cidaas API support. Check the Verification API documentation for current capabilities.

Non-Migratable Verification Methods

These require user re-enrollment:

SMS OTP: Phone numbers must be re-verified (unless email_verified is true)
Email OTP: Email must be re-verified (unless email_verified is true)
Backup Codes: Must be regenerated
FIDO2/WebAuthn: Security keys must be re-enrolled
Pattern/PIN: Must be re-setup
Push Notifications: Must be re-configured

Step 3: Migration Strategy for Verification

Strategy A: Migrate Status Only

For email and mobile verification, migrate the verification status:

Advantages:

Users don't need to re-verify immediately
Smooth user experience
Faster migration

Considerations:

Only migrate if source system verification is trustworthy
Consider re-verification for high-security scenarios

Implementation:

{
  "email": "[email protected]",
  "email_verified": true,  // From source system
  "mobile_number": "+49123456789",
  "mobile_number_verified": true  // From source system
}

Strategy B: Require Re-Verification

Force users to re-verify after migration:

Advantages:

Ensures data accuracy
Confirms user still has access
Higher security

Disadvantages:

Users must complete verification flow
May cause friction

Implementation:

{
  "email": "[email protected]",
  "email_verified": false,  // Force re-verification
  "mobile_number": "+49123456789",
  "mobile_number_verified": false  // Force re-verification
}

Strategy C: Gradual Re-Verification

Migrate status but require re-verification on next login:

Advantages:

Balance between UX and security
Users verify when they naturally log in

Implementation:

Migrate with email_verified: true
Set flag to require verification on next login
User verifies during normal login flow

Migration Planning Checklist

Use this comprehensive checklist to ensure all planning aspects are covered:

Pre-Migration Planning

Technical Preparation

Migration Execution

Next Steps

After completing your migration planning:

Review the Migration Overview for a complete guide to migration procedures and topics
Proceed with User Migration for technical implementation details
Choose your Migration Procedure (one-time, sync, or bidirectional)
Set up your Migration API Access
Begin with a test migration using a small user subset

Table of Contents​

Merging Multiple Data Sources​

Understanding the Challenge​

Step 1: Identify User Identity Keys​

Step 2: Define Merge Strategy​

Strategy A: Source Priority​

Strategy B: Most Recent Wins​

Strategy C: Field-Specific Rules​

Step 3: Handle Duplicate Detection​

Pre-Migration Deduplication​

Matching Algorithms​

Step 4: Create Unified User Structure​

Step 5: Migration Order​

Defining User Attribute Structure​

Step 1: Categorize Attributes​

System Fields (Built-in)​

Custom Fields​

Step 2: Field Mapping Strategy​

Step 3: Data Quality and Validation​

Step 4: Field Setup in cidaas​

Handling Social Providers​

Understanding Social Provider Identities​

Step 1: Identify Social Provider Users​

Step 2: Map Social Provider Data​

Step 3: Migration Strategies for Social Providers​

Strategy A: Migrate as Identities​

Strategy B: Social-Only Users​

Step 4: Account Linking Considerations​

Step 5: Post-Migration Verification​

Handling Verification Methods​

Step 1: Identify Verification Methods​

Step 2: Migratable vs Non-Migratable Methods​

Migratable Verification Methods​

Non-Migratable Verification Methods​

Step 3: Migration Strategy for Verification​

Strategy A: Migrate Status Only​

Strategy B: Require Re-Verification​

Strategy C: Gradual Re-Verification​

Migration Planning Checklist​

Pre-Migration Planning​

Technical Preparation​

Migration Execution​

Next Steps​

Table of Contents

Merging Multiple Data Sources

Understanding the Challenge

Step 1: Identify User Identity Keys

Step 2: Define Merge Strategy

Strategy A: Source Priority

Strategy B: Most Recent Wins

Strategy C: Field-Specific Rules

Step 3: Handle Duplicate Detection

Pre-Migration Deduplication

Matching Algorithms

Step 4: Create Unified User Structure

Step 5: Migration Order

Defining User Attribute Structure

Step 1: Categorize Attributes

System Fields (Built-in)

Custom Fields

Step 2: Field Mapping Strategy

Step 3: Data Quality and Validation

Step 4: Field Setup in cidaas

Handling Social Providers

Understanding Social Provider Identities

Step 1: Identify Social Provider Users

Step 2: Map Social Provider Data

Step 3: Migration Strategies for Social Providers

Strategy A: Migrate as Identities

Strategy B: Social-Only Users

Step 4: Account Linking Considerations

Step 5: Post-Migration Verification

Handling Verification Methods

Step 1: Identify Verification Methods

Step 2: Migratable vs Non-Migratable Methods

Migratable Verification Methods

Non-Migratable Verification Methods

Step 3: Migration Strategy for Verification

Strategy A: Migrate Status Only

Strategy B: Require Re-Verification

Strategy C: Gradual Re-Verification

Migration Planning Checklist

Pre-Migration Planning

Technical Preparation

Migration Execution

Next Steps