Bringing Generative AI to Enterprise Messaging: Jedi Hill's Planned AWS Bedrock Upgrade for INNOWAVE
INNOWAVE, a high-throughput SMS/messaging (CPaaS) platform currently running on Alibaba Cloud, is planning a migration to AWS with a new managed Amazon Bedrock inference layer — designed by Jedi Hill to bring tiered generative AI capabilities to enterprise messaging at scale.
The Challenge
INNOWAVE operates a high-concurrency SMS platform (Spring Boot microservices on Kubernetes, with RocketMQ, MySQL RDS, Tair/Redis, and ADB) connecting to upstream carriers via SMPP/HTTPS, currently hosted on Alibaba Cloud. As the company looks to add generative AI capabilities to its platform, it faces a familiar scaling challenge: how to add LLM-powered features across a high-volume, latency-sensitive messaging workload without applying an expensive, one-size-fits-all model to every request.
The Planned Solution
Jedi Hill is designing a migration to AWS (ap-southeast-2) using a reseller + managed service provider (MSP) model, with a five-tier model routing architecture built on Amazon Bedrock:
Amazon Titan Embeddings — used for retrieval/classification groundwork
Amazon Nova Lite — handles first-line classification and routing decisions
Claude Haiku — handles simple conversational responses
Claude Sonnet — handles complex conversations and RAG (retrieval-augmented generation) queries
Claude Opus — reserved for high-risk escalations requiring the most capable reasoning
All inference requests are routed through Amazon Bedrock Guardrails, ensuring consistent safety and content policy enforcement regardless of which model tier handles a given request.
Infrastructure design: the platform runs on Amazon EKS across four logical namespaces, with all Bedrock access routed through a VPC Interface Endpoint rather than the public internet. The design spans data, security, and observability layers, with planned use of Karpenter for node autoscaling, KEDA for event-driven scaling, IRSA (IAM Roles for Service Accounts) for least-privilege access, and ArgoCD for GitOps-based deployment.
Figure: Planned AWS architecture for INNOWAVE — Amazon EKS across four namespaces, with all Bedrock inference routed through a VPC Interface Endpoint and a five-tier model routing layer (Titan → Nova Lite → Claude Haiku → Sonnet → Opus).
Why a Tiered Approach
Rather than routing every inference request to a single frontier model, the five-tier design sends the majority of traffic — straightforward classification or simple conversation — to lower-cost models (Nova Lite, Claude Haiku), reserving Claude Sonnet and Opus for the smaller share of requests that genuinely require deeper reasoning or RAG-based retrieval. This mirrors the cost/quality tradeoff analysis Jedi Hill has applied across similar generative AI migrations: match model capability to task complexity rather than defaulting to the most expensive model for every request.
Target Outcomes
Based on the business case and technical feasibility modeling completed to date, the project targets (once fully ramped across Phase 1–3):
Approximately $530K implementation build cost
A targeted $1.23M in annual recurring AWS-related revenue
A 15% MSP margin on ongoing managed services
A full cost model comparing tiered routing against a single-frontier-model approach, showing meaningful savings at INNOWAVE's projected volume
About Jedi Hill
Jedi Hill is an AWS Partner specializing in generative AI consulting, cloud migration, and managed services, helping enterprises design cost-efficient, production-grade AI architectures on AWS.
Looking to bring tiered generative AI to a high-volume platform? Contact Jedi Hill to learn more.
Comments