**This is a repost - Post was deleted by mistake
https://lyon-craft.fr/sessions/l-ia-dans-vos-projets-un-peu-beaucoup-pas-du-tout.html
Speaker: Marie-Alice BLETE – 10+ years as Java dev, now specializing in AI –Co author of Developing Apps with ChatGTP publish @ O’Reilly – Currently working at Komodo Health
1. Live AI-Powered Web Interfaces
- Demo of a website shop with AI integration.
- Users can request new features live, which are then added via AI in real time.
- Demonstrates the potential of LLMs to drive interactive, on-demand UX.
2. AI Agents: Autonomy & Challenges
- Agents operate based on objectives (e.g., via LangChain framework):
- Prompt defines goals.
- Agents are given access to tools (e.g., Python function for product search).
- Each tool must be explicitly defined as usable.
- Pros: Can act autonomously.
- Cons:
- Hallucination risks (like any LLM).
- High cost, especially for simple use cases or POCs.
- Unstable behavior due to the evolving nature of LLM providers.
3. Architecture Considerations
-
Traditional: Monolithic “black-box” agents.
- Difficult to maintain and scale.
- Adding new capabilities may disrupt existing ones.
- Requires larger context → higher costs.
-
Alternative: Modular multi-agent systems.
- Each agent has a specific task.
- Allows tracing and debugging.
- Downside: Rigid scenarios → poor generalization outside designed use cases.
4. Development Lessons (Maplab Internal Use Case)
- Example: Drug search system avoids LLM for final output → Uses LLM to generate API queries (like text-to-SQL).
- Production Learnings after 9 months:
- Team composition: Mix of devs, ML engineers, researchers. Slower but more comprehensive.
- Estimations: LLM work is less predictable than traditional dev.
- Testing: Shift from unit testing to system-wide evaluations. Harder to guarantee output quality.
- Tech stack: Constant reevaluation due to rapid evolution of models, providers, capabilities.
- RTE is unreliable due to novelty of the domain and lack of community experience.
5. Q&A Highlights
-
Q: How to prevent hallucinations?
A: No silver bullet. LLMs are designed for completion, not Q&A. RAG (Retrieval-Augmented Generation) helps, but building a reliable system takes time. Plug-and-play doesn’t work. -
Q: Can LLMs be told to stay silent if unsure?
A: Technically no. LLMs don’t “know” what they don’t know. Tools like Inspector Agents can flag hallucinations, but they double the cost. -
Q: ROI of implemented features?
A: Still early. Clients appreciate the chatbot UX (e.g., spreadsheet-powered search), but hard to quantify return. It is a strong marketing differentiator. -
Q: How is data security ensured (e.g., for medical data)?
A: LLMs don’t access data directly. They generate queries sent to secure APIs. The system is designed so that LLMs only act as interpreters, not data processors.