Agentic AI & LLM Application Development
- Design, build, and deploy production-grade AI agents and multi-agent workflows using frameworks such as Google's Agent Development Kit (ADK), LangChain, or equivalent.
- Implement tool use, function calling, and structured outputs, integrating agents with internal APIs, databases, and third-party services (including via the Model Context Protocol).
- Develop and iterate on prompts, context strategies, and orchestration logic; establish version control and review practices for prompts as first-class artefacts.
- Apply appropriate guardrails, input/output validation, and human-in-the-loop checkpoints for agents that take consequential actions.
- Evaluate build-versus-buy trade-offs across model providers, frameworks, and managed services, and make pragmatic recommendations.
Retrieval-Augmented Generation & Knowledge Systems
- Build and maintain RAG pipelines end to end: ingestion, parsing, chunking, embedding, indexing, retrieval, re-ranking, and grounded generation.
- Select and operate vector stores appropriate to the workload (e.g. Vertex AI Vector Search, pgvector on Cloud SQL or AlloyDB, or equivalent).
- Implement hybrid and metadata-filtered retrieval strategies, and tune them against measured retrieval quality rather than intuition.
- Design document processing workflows across heterogeneous and messy source formats, including incremental refresh and deletion handling.
Cloud Engineering & Deployment
- Build, containerise, and deploy AI services and APIs on Google Cloud Platform, primarily using Cloud Run, Cloud Functions, and Vertex AI.
- Design and manage supporting cloud infrastructure — Cloud SQL, BigQuery, Cloud Storage, Pub/Sub, Artifact Registry, Secret Manager — with sound security choices around networking, IAM, and service accounts.
- Manage infrastructure as code (e.g. Terraform) and keep environments reproducible.
- Optimise for cost, latency, and throughput, including caching, batching, streaming responses, and right-sizing model selection to the task.
Data Engineering & Integration
- Build and maintain the data pipelines that feed AI systems, spanning batch and streaming ingestion from APIs, databases, and file sources.
- Write and optimize SQL against BigQuery and relational databases for both application queries and analytical workloads.
- Model and manage the operational data layer (Cloud SQL, Firestore, or similar) supporting agent state, session history, and application data.
- Implement data quality checks, schema management, and lineage where it materially affects downstream AI behaviour.
Evaluation, Monitoring & Observability
- Design and implement evaluation frameworks for LLM and agent systems, including golden datasets, offline eval suites, LLM-as-judge scoring, and regression testing across prompt and model changes.
- Instrument applications with tracing and structured logging (e.g. OpenTelemetry, Cloud Trace, Langfuse, Arize Phoenix, or equivalent) to make agent behaviour debuggable.
- Build monitoring and alerting for quality, latency, error rates, token consumption, and cost, and act on degradation proactively.
- Track groundedness, hallucination, and refusal behaviour in production, and close the loop between observed failures and system improvements.
- Investigate and troubleshoot AI-related incidents in production, including non-deterministic and hard-to-reproduce failures.
Python & Software Engineering Practice
- Write clean, tested, maintainable Python; contribute to shared libraries and internal tooling.
- Work within Git-based collaborative development practices, including branching strategies, pull requests, and code review.
- Apply sound API design, dependency management, and packaging practices to AI services.
- Balance rapid prototyping with the discipline required to make prototypes production-ready.
Documentation & Enablement
- Maintain clear documentation for AI systems, architectures, evaluation results, and operational runbooks.
- Communicate capabilities, limitations, and risks of AI systems honestly to technical and non-technical stakeholders.
- Support colleagues and clients in adopting the systems you build.
