Development with local LLMs: Privacy & Performance in Practice

What if you didn't have to send your code and other sensitive data to the cloud when working with AI? Today, you can run language models directly on your own computer. It has its pitfalls, but the capabilities of local LLMs have advanced significantly in recent months.

At the workshop, you will find out when local AI makes sense and how to incorporate it into your development practice. We will show you how to choose a suitable model and how to navigate the trade-offs between its quality, speed, memory requirements and hardware price. We will also look at the differences between inference engines and what hardware to buy so that you don't overpay or run into its limits.

We will also focus on selecting and setting up tools that connect LLMs with the development workflow. We will explain how working with local models differs from cloud services and how to adapt prompts to their capabilities and limitations.

The workshop will be practical, illustrative and open to discussion. We will test local models on specific examples.

The workshop is intended for developers and other technically oriented participants who want to gain more control over their data and AI tools.

Requirements for participants:

• Previous experience with running their own models is not necessary.
• Not every laptop has sufficient power, so the lecturer will show you things at home or in the cloud. Alternatively, we will try to run a rented GPU.