How to Evaluate a Dev Partner: 12 Questions That Expose the Generalists

You found an agency. Their portfolio looks strong. They have built apps, platforms, SaaS tools. They say they have "done healthcare before." The proposal arrives quickly. The price seems right.

According to a CB Insights post-mortem analysis of failed startups, 23% cite team and execution failures as a primary cause of failure, and for HealthTech specifically, execution failure almost always has a compliance or technical architecture dimension behind it.

The issue is not that generalist agencies are bad at building software. They are often very good. The issue is that healthcare software is a different category, with different risk surfaces, different regulatory requirements, and different failure modes.

And the questions most founders ask during agency selection are not designed to expose that gap. These 12 questions are as follows.

Understanding What You Are Actually Buying

When you hire a dev partner for a HealthTech product, you are not just buying code. You are buying:

• A compliance posture that either opens or closes hospital doors
• A technical architecture that either passes or fails investor due diligence
• A documentation trail that either satisfies or fails regulatory submissions
• A codebase that either holds up under a security audit or falls apart

A generalist agency can deliver good code. What they typically cannot deliver is a product that is clinically credible, compliance-ready, and investor-auditable from day one.

The questions below are designed to surface that gap quickly.

12 Questions (And What Good Answers Look Like)

Overview

• Questions 1 to 4: Proving Real HealthTech Experience
• Questions 5 to 8: Architecture, Security, and Interoperability
• Questions 9 to 12: Process, Ownership, and Red-Flag Patterns

Question 1: Can you show me a HealthTech product you have built that is live in a hospital or clinical environment?

This is the most basic filter and it eliminates more agencies than you would expect.

Good answer: A named product, a named hospital or health system, a description of the clinical workflow it supports, and ideally a reference contact

Red flag answer: "We have built several healthcare-adjacent apps" or "we worked with a wellness startup" wellness and fitness apps are not clinical healthcare

What it tells you: Hospitals have procurement processes, security reviews, and integration requirements that wellness apps do not. A team that has shipped into a clinical environment has faced real constraints. A team that has not is learning on your time

Question 2: Walk me through how you handled HIPAA compliance on your last HealthTech project. What did your BAA process look like and who owned the audit trail?

If the answer is "we followed best practices" without specifics, that is your answer.

Good answer: A clear description of Business Associate Agreement (BAA) structure, which subprocessors were involved, how PHI was segregated, and who owned the compliance documentation

Red flag answer: "We used AWS which is HIPAA compliant" (this is the most common and most misleading answer in HealthTech dev discussions. AWS eligibility is a starting point, not a compliance posture)

What it tells you: HIPAA compliance is an organizational and architectural discipline, not a checkbox. Teams who understand it can explain it in plain language

Question 3: Have you built anything that required GDPR compliance, DTAC assessment in the UK, or health data regulations in the UAE or Saudi Arabia?

This matters enormously if you are building for the UK, GCC, or European markets.

Good answer: Specific experience with the relevant framework, including what changed in their architecture or documentation process as a result

Red flag answer: "GDPR is basically the same as HIPAA" (it is not, and saying so signals a shallow understanding of both)

What it tells you: Regulatory requirements in digital health vary significantly by market. A partner who only understands US rules is a liability the moment you run a pilot in Abu Dhabi or London

Question 4: Have you built a product that qualified as Software as a Medical Device (SaMD)? If so, what standard did you follow for software lifecycle documentation?

This question specifically targets Medical Device Manufacturers, CROs, and anyone building AI-assisted clinical tools.

Good answer: Clear reference to IEC 62304 (software lifecycle), ISO 14971 (risk management), and a description of how they structured their design history file or software development plan

Red flag answer: "We can document whatever standard you need" without prior experience applying it

What it tells you: IEC 62304 and ISO 13485 compliance are not documentation styles you can learn as you go. They require specific engineering discipline from the first line of code. A team encountering these standards for the first time on your project will cost you 6 to 12 months of rework

Question 5: How do you handle EHR integration? Have you connected to Epic, Cerner, or any regional EHR system in the Middle East such as Malaffi or Wareed?

Good answer: Direct experience with at least one major EHR integration, knowledge of FHIR R4 as the current interoperability standard, and an honest account of the timeline and complexity involved

Red flag answer: "We can integrate with any system" without naming one they have actually connected to

Question 6: How do you approach security architecture for a product handling Protected Health Information? What does your threat modelling process look like?

Good answer: A description of their security design process, including access control design, encryption standards, audit logging, and how they approach penetration testing before clinical launch

Red flag answer: "We follow security best practices and use established cloud providers" without any specifics

What it tells you: Hospital IT teams run formal security assessments before onboarding any third-party software. If your dev partner has never built for that level of scrutiny, your product will fail those assessments

Question 7: How do you handle data residency requirements for products deployed in the UAE or Saudi Arabia?

For GCC-focused founders, this question alone will filter out most agencies.

Good answer: Knowledge of NHIC requirements in KSA, MoHAP and DHA requirements in UAE, and a clear view of which cloud regions and data processing agreements are needed

Red flag answer: "We use AWS Middle East" without specifying which zone or understanding the difference between Bahrain and in-country UAE or KSA regions

What it tells you: Getting data residency wrong in GCC healthcare is not a minor issue. It can void hospital contracts and expose your company to significant regulatory penalties

Read More: AI Use in the GCC: Hosting Clinical Models Locally in KSA and UAE

Question 8: What is your approach to clinical data modelling? Have you worked with HL7 FHIR, SNOMED CT, ICD-10, or LOINC?

Good answer: Direct experience mapping clinical data to at least one of these standards, with a clear explanation of why standardised clinical terminology matters for interoperability and regulatory submissions

Red flag answer: "We can learn whatever data format you need"

What it tells you: Clinical data is not generic data. A team that has never worked with clinical terminologies will produce a data model that cannot connect to hospital systems or satisfy regulatory reviewers

Question 9: What does your typical discovery process look like for a HealthTech product, and how do you handle compliance requirements during that phase rather than after?

This is one of the most revealing questions in the list.

Good answer: A structured discovery process that includes regulatory scoping, security architecture decisions, and compliance framework selection before the first sprint begins

Red flag answer: "We start with a prototype and add compliance layers later"

What it tells you: In HealthTech, retrofitting compliance onto an existing codebase is always more expensive and often impossible without a full rebuild. Teams that treat compliance as a phase 2 activity are teams that have not built regulated products

Question 10: If a hospital IT team ran a security audit on the code you delivered, what would they find?

Ask them to answer this honestly. The pause before the answer is informative.

Good answer: A description of their security baseline, what they regularly include (encrypted data at rest, audit logs, role-based access control, secure secret management, no PHI in logs), and what they would recommend the client address before a formal audit

Red flag answer: Confident assurance with no specifics, or immediate deflection to "we follow OWASP guidelines"

Question 11: Who on your team has clinical domain knowledge? Not just technical, but actual understanding of clinical workflows?

Good answer: Named people with backgrounds in clinical informatics, nursing, medicine, or health operations who are involved in product decisions, not just in sales conversations

Red flag answer: "Our team learns the domain on each project" or "we have worked with clinicians as stakeholders"

What it tells you: Clinical workflow knowledge is not something that transfers from reading a requirements document. Teams without it consistently build products that clinical staff find unusable or that create patient safety risks

Question 12: Can you describe a project that did not go well and what happened?

Every team has a story like this. How they tell it tells you everything.

Good answer: A clear, honest account of what went wrong, what they learned, and what changed in their process as a result

Red flag answer: Inability to name one, or a story that positions all blame externally

How to Score Your Shortlist

Use this table to evaluate up to three agencies against the questions above.

Score each on a 1 to 3 scale: 1 for a weak or vague answer, 2 for a partially satisfactory answer, 3 for a strong specific answer. Any agency scoring below 2 on the four High-weight questions should be removed from consideration regardless of their total score.

Patterns That Should Stop a Conversation Immediately

Beyond the individual questions, watch for these broader signals during agency conversations:

Fast Proposal

A detailed proposal arriving within 24 to 48 hours of your first conversation is almost always a recycled template. A team that understands HealthTech knows that proper scoping takes time. Speed here is not efficiency; it is a sign they are not thinking carefully about your specific requirements.

General Portfolio

Agencies with portfolios covering e-commerce, gaming, education, and healthcare are generalists. Breadth is not depth. The skills required to build a HIPAA-compliant RPM platform or an IEC 62304-compliant SaMD are not transferable from building a retail app.

Compliance as a Service Add-on

If a partner frames compliance as something you can "add on" after the MVP is built, they do not understand HealthTech. Compliance in regulated healthcare software is an architectural decision made before the first line of code, not a layer applied at the end.

No reference You can Call

Every competent HealthTech dev partner should be able to provide at least one reference contact at a live product in a clinical setting. If they cannot, or if they cite confidentiality as the reason for all of them, that is a pattern worth questioning.

Read More: Specialist vs. Generalist: Evaluating a HealthTech Partner

Quick Reference: What Generalists Say vs. What Specialists Say

Conclusion

Choosing a dev partner for a HealthTech product is not a standard vendor decision. The wrong choice does not just slow you down. It can cost you a hospital contract, a funding round, or months of rebuild work you did not budget for.

The 12 questions in this guide are not designed to be difficult. They are designed to surface one simple truth: has this team actually done this before, in a regulated clinical environment, with real consequences if they got it wrong?

Ask them. Listen carefully to the answers. The right partner will not hesitate.