We’re thrilled to be a Platinum Sponsor of EuroPython again this year, happening from July 17th-23rd in Prague, Czechia. If you can’t make it in person, you can still attend the conference remotely, as EuroPython is a hybrid event this year!

Our team member: Steve Dower, will be giving talks during the conference:

• Python & Visual Studio Code, Copilot – Revolutionizing the way you do data science.
• CPython Developer Panel.

Make sure you don’t miss them!  We’ll also be at the Microsoft booth talking about the hard work our teams have been doing to continue supporting the Python community, and to improve the experience for Python developers across our products over the past year.

For instance, Visual Studio Code, which has become the most used editor for Python developers in 2021, has a new and improved experience when working with Jupyter notebooks, a revamped test explorer via the Python extension, and now comes with new Python dev tools extensions (such as pylint, black and isort). We have also enabled a smooth and lightweight editing experience on the web with vscode.dev and github.dev, and a feature-rich one in GitHub Codespaces.

We have also improved Data Science workflows inside of Visual Studio Code. Polyglot notebooks now supports Python, allowing you to use multiple programming languages natively all in the same notebook in Visual Studio Code! No more needing wrapper libraries or magic commands to work with your favorite languages in the same notebook! With polyglot notebooks, each language in Polyglot Notebooks gets a first-class editing experience with language server support such as autocompletion, syntax highlighting, and signature help. Variable explorer now allows you to check values and share variables across all the supported languages.

Additionally, we have launched the Data Wrangler extension in Visual Studio Code. With Data Wrangler, you can seamlessly clean and explore your data in VS Code. It offers a variety of features that will help you quickly identify and fix errors, inconsistencies, and missing data. You can perform data profiling and data quality checks, visualize data distributions, and easily transform data into the format you need. Plus, Data Wrangler comes with a library of built-in transformations and visualizations, so you can focus on your data, not the code. As you make changes, the tool generates code using open-source Python libraries for the data transformation operations you perform. This means you can write better data preparation programs faster and with fewer errors. The code also keeps Data Wrangler transparent and helps you verify the correctness of the operation as you go.

In the machine learning and AI realm, Azure Machine Learning Visual Studio Code Web integration is now in public preview. VS Code for the Web provides you with a full-featured development environment for building your machine learning projects, all from the browser and without required installations or dependencies. And by connecting your Azure Machine Learning compute instance, you get the rich and integrated development experience VS Code offers, enhanced by the power of Azure Machine Learning. Furthermore, Prompt Flow for Azure Machine Learning is now in preview! Prompt Flow provides a streamlined experience for prompting, evaluating, tuning, and operationalizing large language models. With prompt flow, you can quickly create prompt workflows that connect to various language models and data sources. This allows for building intelligent applications and assessing the quality of your workflows to choose the best prompt for your case.

These are only some of the many things we look forward to chatting with you at EuroPython. But if you can’t make it there, no problem - you can always connect to us through our Discord channel.

Whether it’s in person or virtually, we all look forward to meeting you!

By David Seddon from Kraken Technologies.

Hi, I’m David, a Python developer at Kraken Technologies. I work on Kraken: a Python application which has, at last count, 27,637 modules. Yes, you read that right: nearly 28k separate Python files - not including tests. I do this along with 400 other developers worldwide, constantly merging in code. And all anyone needs to make a change - and kick start a deployment of the software that runs 17 different energy and utility companies, with many millions of customers - is one single approval from a colleague on Github.

Now you may be thinking this sounds like a recipe for chaos. Honestly, I would have said the same. But it turns out that large numbers of developers can, at least in the domain we work in, work effectively on a large Python monolith. There are lots of reasons why this is possible, many of them cultural rather than technical, but in this blog post I want to explain about how the organisation of our code helps to make this possible.

Layering our code base

If you’ve worked on a code base for any length of time, you will have felt the drift towards unpleasant complexity. Strands of logic tangle together across your application, and it becomes increasingly difficult to think about parts of it in isolation. This is what started happening to our young code base, and so we decided to adopt what is known as a ‘layered architecture’ where there are constraints about what parts of the code base can know about each other.

Layering is a well-known software architecture pattern in which components are organized, conceptually, into a stack. A component is not allowed to depend on any components higher up the stack.

For example, in the above diagram, C would be allowed to depend on B and A, but not D.

The idea of a layered architecture is broad: it may be applied to different kinds of components. For example, you could layer several independently-deployable services; or alternatively your components could just be a set of source code files.

What constitutes a dependency is also broad. In general, if a component has direct knowledge of another component (even if purely at a conceptual level) then it depends on it. Indirect interaction (e.g. via configuration) is not usually seen as a dependency.

Layers in Python

In a Python code base, the layers are best thought of as Python modules, and dependencies as import statements.

Take the following code base:

myproject
    __init__.py
    payments/
        __init__.py
        api.py
        vendor.py
    products.py
    shopping_cart.py

The top-level modules and subpackages are good candidates for layers. Let’s say we decide the layers should be in this order:

shopping_cart
payments
products

Our architecture would thus forbid, for example, any of the modules within payments from importing from shopping_cart. They could, however, import from products.

Layering can also be nested, so we could choose to layer within our payments module like so:

api
vendor

There’s no single, correct way of choosing which layers exist, and in which order - that’s an act of design. But layering like this leads to a less tangled code base, making it easier to understand and change.

How we’ve layered Kraken

At the time of writing, 17 different energy and utility companies license Kraken. We call these companies clients, and run a separate instance for each. Now, one of Kraken’s main characteristics is that different instances are ‘the same, but different’. In other words, there is a lot of shared behavior, but also every client has bespoke code that defines their specific needs. This is also true at the territory level: there are commonalities between all the clients that run in Britain (they integrate with the same energy industry) that aren’t shared with, say, Octopus Energy Japan.

As Kraken grew into a multi-client platform, we evolved our layering to help with this. Broadly speaking, it now looks like this at the top level:

kraken/
    __init__.py
    
    clients/
        __init__.py
        oede/
        oegb/
        oejp/
        ...
    
    territories/
    	__init__.py
        deu/
        gbr/
        jpn/
        ...
        
    core/

The clients layer is at the top. Each client gets a subpackage inside that layer (for example, oede corresponds to Octopus Energy Germany). Below that is territories, for all the country-specific behaviour, again with territory-specific subpackages. The bottom layer is core, which contains code that is used by all clients. There is an additional rule, which is that client subpackages must be independent (i.e. not import from other clients), and the same goes for territories.

Layering Kraken like this allows us to make changes with a limited ‘blast radius’. Because the clients layer is at the top, nothing depends on it directly, making it easier to change something that relates to a particular client without accidentally affecting behavior on a different client. Likewise, changes that relate only to one territory won’t affect anything in a different one. This allows us to move quickly and independently across teams, especially when we are making changes that only affect a small number of Kraken instances.

Enforcing layering with Import Linter

When we introduced layering, we quickly found that just talking about the layering was not enough. Developers would often accidentally introduce layering violations. We needed to enforce it somehow, and we do this using Import Linter.

Import Linter is an open source tool for checking that you are following layered architectures. First, in an INI file you define a contract describing your layering - something like this

[importlinter:contract:top-level]

name = Top level layers
type = layers
layers =
    kraken.clients
    kraken.territories
    Kraken.core

We can also enforce the independence of the different clients and territories, using two more contracts (this time `independence` contracts)

[importlinter:contract:client-independence]
name = Client independence
type = independence
layers =
    kraken.clients.oede
    kraken.clients.oegb
    kraken.clients.oejp
    ...

[importlinter:contract:territory-independence]
name = Territory independence
type = independence
layers =
    kraken.territories.deu
    kraken.territories.gbr
    kraken.territories.jpn
    ...

Then you can run lint-imports on the command line and it will tell you whether or not there are any imports that break our contracts. We run this in the automated checks on every pull request, so if someone introduces an illegal import, the checks will fail and they won’t be able to merge it.

These are not the only contracts. Teams can add their own layering deeper in the application: kraken.territories.jpn, for example, is itself layered. We currently have over 40 contracts in place.

Burning down technical debt

When we introduced the layered architecture, we weren’t able to adhere to it from day one. So we used a feature in Import Linter which allows you to ignore certain imports before checking the contract.

[importlinter:contract:my-layers-contract]
name = My contract
type = layers
layers =
    kraken.clients
    kraken.territories
    kraken.core
ignore_imports =
    kraken.core.customers ->
    kraken.territories.gbr.customers.views
    kraken.territories.jpn.payments -> kraken.utils.urls
    (and so on...)

We then used the number of ignored imports as a metric for tracking technical debt. This allowed us to observe whether things were improving, and at what rate.

Here’s our graph of how we’ve been working through ignored imports over the last year or so. Periodically I share this to show people how we’re doing and encourage them to work towards complete adherence. We use this burndown approach for several other technical debt metrics too.

Downsides, there are always downsides

Local complexity

At some point after adopting a layered architecture, you will run into a situation where you want to break the layers. Real life is complex, there are interdependencies everywhere, and you will find yourself wanting to, say, call a function that’s in a higher layer.

Fortunately, there is always a way around this. It’s called inversion of control and it’s easy to do in Python, it just requires a mindset shift. But it does lead to an increase in ‘local’ complexity (i.e. in a little part of your code base). However, it’s a price worth paying for a simpler system overall.

Too much code in higher layers

The higher the layer, the easier the change. We deliberately made it easy to change code for specific clients or territories. Code in the core, which everything depends on, is more costly and risky to make changes to.

As a result, there has been a design pressure, brought about partly by the layering we chose, to write more client and territory-specific rather than introduce deeper, more globally useful code into the core. As a result, there is more code in the higher layers than we might ideally like. We’re still learning about how to tackle this.

We’re still not finished

Remember those ignored imports? Well, years on, we still have some! At last count, 15. Those last few imports are the stubbornest, most tangled ones of all.

It can take serious effort to retrospectively layer a code base. But the sooner you do it, the less tangling you’ll have to address.

In summary

Layering Kraken has kept our very large code base healthy and relatively easy to work with, especially considering its size. Without imposing constraints on the relationships between the tens of thousands of modules, our code base would probably have tangled into an enormous plate of spaghetti. But the large scale structure we chose - and evolved along with the business - has helped us work in large numbers on a single Python code base. It shouldn’t be possible, but it is!

If you’re working on a large Python codebase - or even a relatively small one - give layering a try. The sooner you do, the easier it will be.

Kraken Technologies LTD's is sponsor of EuroPython 2023, check them out on https://kraken.tech/

Hey there!

Springtime is upon as, with ~100 days to the conference we have a lot of updates to share this month regarding our ticket sales, CFP, a new keynoter, and sponsorship as we get closer to the conference.

🇨🇿 Picture a Pythonic Prague

To be a part of EuroPython is to be a part of something greater than the sum of its parts. Our community is fortunate to include a wealth of talent and a diversity of skills, and nowhere is this more obvious than in the design and production of the website.

We’re hugely thankful for the work of Patrick and Raquel in coordinating the brand new design, look and feel for this year’s iteration of our presence on the web.

EuroPython’s peripatetic conference journey through the cultural capitals of Europe is an opportunity for us to celebrate our diverse heritage, history and cultures. This year’s design reflects our Bohemian location in the Czech Republic and we hope you enjoy finding subtle (and not so subtle) aspects of the website that acknowledge and respect our wonderful host city of Prague.

Go check it out! https://ep2023.europython.eu/

📣 Programme

Our CFP finished on  Sunday, 26 March 2023 and we received a record breaking 556 proposals beating 2022 by a large margin. 🎉

Community Voting is currently underway. We invite all eligible voters to cast your votes  by Friday 14 April AoE and show us what YOU would like to see at EuroPython 2023! Your vote brings a plurality and diversity of voices to the decision making process; and your feedback is an important ingredient in this refinement and curation process. In 2022, we had a total of 24,000 votes and hope to reach 40,000 votes this year. Cast your votes here: https://ep2023.europython.eu/voting

We have also begun the panel review of the proposals. We received almost 100 reviewer applications and are grateful and overjoyed with the community’s enthusiasm and support for sharing their insights and expertise!

🗣️ Keynote announcement - Sophie Wilson

We are delighted and honoured to announce Sophie Wilson has agreed to keynote at this year's EuroPython. It is not an understatement to say that every one of us, whether we realise it or not, has benefitted from Sophie's contributions to the field of computing.

Since her teenage years, Sophie has designed and built microprocessor based systems. Early projects included a system for counting translucent drops of liquid and detecting spun fibre machinery breakdowns; while in her first university vacation she developed an automated cow-feeder.

After university she joined Acorn Computers Ltd, where she designed the Acorn System 1, coding the operating system in binary before designing and implementing Acorn Assembler, Acorn MOS and BASIC. She, and her Acorn colleague Steve Furber, took less than a week to design and implement the prototype of the BBC Microcomputer, winning Acorn the contract for the BBC's Computer Literacy Project. Sophie designed the operating system and designed and implemented BBC BASIC for a succession of processors. Anyone alive in Europe in the 1980s will have encountered her work, such was the ubiquity of the BBC Microcomputer, and BBC BASIC is rightly celebrated as an example of a powerful programming language that is also easy for beginner coders to learn (not unlike another programming language with which we're all familiar).

She and Furber went on to co-design the ARM processor, powering Acorn's computers during the 1990s and virtually every mobile phone and tablet in the world today – 200 billion sales of ARM powered chips (as of July 2022). Acorn's CEO at the time, Hermann Hauser, recalls that "while IBM spent months simulating their instruction sets on large mainframes, Sophie did it all in her brain." MicroPython, CircuitPython and Snek all target ARM processors, and most modern Apple Macs run on chips based upon the ARM design. There's a good chance you're reading this announcement on a machine running an ARM processor based on Sophie’s design.

As if that were not enough, as a founder at Element 14, Sophie went on to develop the Firepath processor, widely used in the telecommunications industry. Broadcom acquired Element 14 in 2000.

Sophie is a Broadcom Fellow and Distinguished Engineer, a Fellow of the Royal Society, a Fellow of the Royal Academy of Engineering, a Distinguished Fellow of the British Computer Society, a Fellow of the Women’s Engineering Society, an honorary Fellow of Selwyn College, Cambridge, an honorary Fellow of the Institution of Engineering and Technology (IET) and an honorary Fellow of the Institution of Engineering Designers (HonFIED). She has an honorary doctorate of science from Cambridge University and is a Commander of the British Empire (CBE).

👩‍🏫 Speaker Mentorship

We ran a One-to-One Speaker Mentorship Programme to support our first time speakers and anyone else wanting support and representation at EuroPython. We successfully  matched all 37 mentees with a mentor. Thank you, every mentor who is giving back to the community!

We also ran an Ask Me Anything workshop about the CFP as part of the Mentorship Programme. In case you missed it, you can catch up here: EuroPython 2023 Mentorship Programme - Ask me Anything about the CFP:

Speaker Placement Programme

For most new speakers, speaking at a conference for the first time can be a bit intimidating. We also understand that not all of our mentees make it to EuroPython. Part of the support we would like to provide to our mentees is to help them find opportunities to speak at a local event or meetup.

If you are an event/ meetup organiser looking for speakers, please fill in the form and we would be happy to introduce our mentees to you if we believe there’s a match.

🕸️ WASM Summit

WebAssembly (abbreviated to WASM) is an important new open technology: a binary instruction format for a virtual machine. Think of it as a new, secure and performant portable compilation target that runs both in browsers and elsewhere the virtual machine runs.

Python (thanks to Pyodide) and MicroPython can both be compiled to WASM, and with the advent of projects like PyScript, Zython and others, Python is making important and innovative inroads into the world of WebAssembly.

In a first for EuroPython, we’re working with members of the community to run a summit that aims to bring together maintainers and users of Python with WebAssembly. The summit is a place to discuss the state of this ecosystem, existing challenges and ongoing work. If you are attending EuroPython and would like to join the summit, check out the agenda and registration details here: https://ep2023.europython.eu/wasm.

🎫 Registration Launched

We have different ticket types and tiers to choose from. We worked really hard this year to lower the ticket price and make them more affordable. In addition, we are also offering Financial Aid  for folks who need extra help to cover tickets, travel and visa costs.

We are again providing free childcare to those who need it! In addition, the sprint weekend will take place in a different venue and is completely open to the public! Stay tuned for more information!

💶 Financial Aid

Our Financial Aid program is in full swing! We’ve already received over 60 applications from over 30 different countries. If you need financial assistance to visit EuroPython, the deadline for the first round of applications is 23 April 2023.

For more information about our Financial Aid Program and our selection criteria, please visit https://europython.eu/finaid.

💸 Call for Sponsors

Big shoutout to our first three confirmed sponsors Numberly, energy & meteo systems and Kraken Technologies LTD! Special shoutout to Numberly, celebrating their 10th anniversary of being a EuroPython sponsor! Thank you for your continuous support, your dedication to the community and the kindness fun (and snakes!) you have brought to our conferences! We cannot wait to see you at your booth!

We are privileged to have many other fantastic companies who have expressed interest in sponsoring EuroPython this year. Apart from the standard sponsor packages, there are many other ways you can support the conference: be a childcare or Financial Aid sponsor, help us with our endeavour to open our spirit days to the public!

If you are interested in sponsoring EuroPython 2023, head to https://ep2023.europython.eu/sponsor and dig into the details. If you still have questions, write to us at sponsoring@europython.eu

🐍 Upcoming Events

PyCon DE & PyData Berlin https://2023.pycon.de/ 🇩🇪
17. April - 19. April 2023

PyCon US https://us.pycon.org
19 - 27 April 2023

PyCon LT https://pycon.lt/2023 🇱🇹
17 - 20 May 2023

PyCon Italia https://pycon.it/en 🇮🇹
25 - 28 MAY 2023

PyCon PL https://pl.pycon.org 🇵🇱
29 June - 02 July 2023

PyCon Taiwan https://tw.pycon.org/2023/en-us 🇹🇼
Sep. 2 - 3 2023

PyCon Estonia https://pycon.ee/ 🇪🇪
Sep. 7 - 8 2023
Call for papers deadline: 21st April, 2023

💥 Open Source Project Feature

MyHDL: From Python to Silicon! https://www.myhdl.org/
MyHDL is a powerful tool that can help you design hardware with Python, It gives you the ability to convert your designs automatically to both Verilog and VHDL and provides hardware engineers with the power of the Python ecosystem

There is a nice talk from PyCon Taiwan 2013 by Jan Decaluwe about using MyHDL to design digital hardware with Python.

🤭 PyJok.es

$ pip install pyjokes
Collecting pyjokes
Downloading pyjokes-0.6.0-py2.py3-none-any.whl (26 kB)
Installing collected packages: pyjokes
Successfully installed pyjokes-0.6.0
$ pyjoke

.NET was named .NET so that it wouldn't show up in a Unix directory listing.