With thousands of delivery partners operating on the ground every day across over 16,000 PIN codes, ensuring that the right person is behind each delivery has become a top priority for Shadowfax. This expansive scale enables us to complete millions of deliveries every day. Still, it also comes with a critical responsibility, ensuring that the person making each delivery is the one verified by us. As we scaled, it became clear that traditional verification methods, like photo IDs or manual KYC checks, weren’t accurate in preventing impersonation or account misuse.

To meet this challenge, we needed a real-time, technology-driven solution that could keep up with the speed and scale of our operations. That’s what led us to build our AI-driven Selfie Validation System, designed to accurately verify that every rider checking in is the right person, thereby reducing fraud and creating a safer, more accountable, and reliable last-mile experience for both businesses and customers.

What is a Selfie Validation System?

A Selfie Validation System uses face recognition technology to verify a person's identity by comparing their live selfie (taken during check-in) with the original photo submitted during onboarding. In the logistics sector, this helps us confirm that a registered delivery partner is the one logging in, preventing impersonation and ensuring secure operations.

The Challenges in Gig-Based Identity Verification

In the gig economy, delivery partners are often onboarded remotely and operate independently. While this model is highly scalable, it’s also vulnerable to fraud:

• Impersonation fraud: Individuals logging in using someone else’s credentials and photo
• Spoofing: Printed selfies or screen images used to trick facial recognition systems
• Account takeovers: Genuine accounts being misused or shared

Standard photo verification had its limitations when it came to spotting sophisticated spoofing attempts. While internal checks helped identify a few cases, we knew we could do more. It became essential to build a technology-first solution that was:

• Real-time and accurate
• Resilient to poor lighting, camera angles, or motion blur
• Smart enough to tell real humans from spoof images

This led to the development of our new, deeply engineered Selfie Validation Pipeline.

Why Verified Delivery Partners Matter

Non-verified delivery partners can create a number of challenges. From the business side, there's a lack of accountability, increased delivery delays, and inconsistency in service, all of which can impact brand trust. For customers, risks include wrong deliveries, potential package tampering, and safety concerns during doorstep interactions.

Having a verified fleet makes a measurable difference:

• Ensures ID-verified and background-checked personnel
• Promotes reliable, timely deliveries
• Improves customer communication and tracking
• Protects sensitive data and goods
• Builds long-term trust in the delivery experience

How the Selfie Validation Model Works

The system went through multiple iterations to address real-world edge cases. Below is a breakdown of the core problems we tackled and how we solved them.

Problem 1: Face Matching with Raw Pixel Comparison Was Unreliable

Initial Approach: We started by training a ResNet18-based classifier to predict if two selfies (onboarding and daily check-in) were from the same person. On a controlled dataset, the model hit 93% accuracy. But real-world performance dropped sharply.

Why It Failed:

• Poor selfie framing
• Low lighting or excessive brightness
• Face blurs due to motion

Our Fix: We moved from raw image comparison to facial embeddings using the face_recognition library (with dlib backend). This gave us:

• More reliable face detection
• Consistent 128-dimensional embeddings
• Improved generalisation to messy, real-world selfies

Problem 2: 128-Dimensional Embeddings Were Not Precise Enough

Initial Setup: MobileFaceNet + cosine similarity (128-D)

Result: Only 83% precision

Issue: The low-dimensional embeddings couldn’t capture enough facial detail.

Our Fix: Switched to FaceNet with 512-dimensional embeddings

Result: Precision jumped from 83% → 96%

Problem 3: Spoof Images Still Got Through

Even with strong face matching, our system would sometimes accept spoofed inputs (printed photos, selfies displayed on screens). That’s because facial structure alone isn’t enough. We needed to check: is this person live?

Our Solution: Build a dedicated liveness detection model

Liveness Model Evaluation

• Tried ResNet18: Great accuracy, but slow (~300–500ms/image)
• Choose MobileNetV2: Balanced speed and accuracy

Final Setup:

• Trained on 15,000+ real and spoofed images
• Added synthetic noise: blur, lighting variations, reflections
• Inference speed: ~80ms per image
• Precision threshold 0.9: 92%

Now, the system could confidently reject static image attacks.

Results & Impact:

The AI-powered selfie validation model has brought significant improvements:

• Face match precision: 96% +
• Liveness detection precision: 92%
• Reduced fraud from spoofed images and impersonation
• Real-time checks at scale with minimal latency
• Boosted trust with clients and platform users

What’s Next: Scaling & Future Enhancements

To make our Selfie Validation system even more reliable and scalable, we’re working on several tech-driven enhancements under the SF Shield umbrella:

1. Full-Day Monitoring & Smarter Check-Ins: To make verification even more reliable, we plan to capture selfies at multiple points during a rider’s shift, not just at login. If the system spots something unusual, like a change in location or device, it will automatically trigger a quick selfie check to confirm everything is in order.
2. On-Device Liveness Detection: We’re optimising our liveness model to run directly on delivery partners’ phones (even in areas with low internet). This will make the process faster and remove the need to call an external API.
3. Learning from Manual Overrides: Whenever our ops team manually approves or flags a selfie, that decision will feed back into the system. This helps us keep improving our model using real-world examples.
4. Adding More Identity Signals: We’re exploring multi-modal checks like voice recognition or gestures (like blinking or nodding) to strengthen verification, especially for high-value or sensitive deliveries.
5. Location-Based Cross-Checks: The system will compare the rider’s selfie location with expected delivery areas using geofencing. This helps detect fake GPS or remote login attempts.
6. City-Level Monitoring Dashboard: We’re building a central dashboard that will help city and cluster leads monitor key metrics like the percentage of verified check-ins, fraud detection heatmaps, frequently flagged users, and manual override trends. This will enable teams to take quick, data-driven decisions and ensure consistent rider verification across all regions.

Looking Ahead

Our Selfie Validation model is more than a security upgrade—it’s a crucial part of our mission to build a safer and more accountable last-mile ecosystem. As gig platforms scale, building trust through verified identities is no longer optional—it’s foundational.

At Shadowfax, we’re committed to engineering solutions that don’t just work in the lab—but thrive in the real world.

FAQs

1. What is selfie validation?

It’s the process of verifying someone’s identity by comparing their live selfie with a previously submitted photo.

2. Why is liveness detection important?

Liveness checks ensure that the selfie is real and not just a photo or screen image, helping prevent spoofing.

3. How does Shadowfax’s face verification system work?

It uses high-accuracy face matching and AI-driven liveness detection, supported by a manual review layer for flagged cases.

4. Can spoof images trick the system?

Not anymore—our liveness model is trained to detect spoof attempts with 92% precision.

5. How is this system helping in the gig economy?

It ensures that every rider logging in is a verified person, improving safety, transparency, and trust.