With 110 days remaining until the big day, the EuroPython programme team is working full steam ahead to put together a power-packed schedule. And what *YOU* want to see at the conference is our guiding light in the process.
Hey ya 👋 hope you are having a good start to 2024 ❤️
It's been a dog’s age since we last spoke. In case you don’t know the reason for our hiatus: we took Elmo’s question of “How is everybody doing?” to its core and had to face the inner truths of our existence.
Luckily, soon the days got longer and the word news got worse so we had to unlock our doors and come out to organise the bestest of conferences!
We are now pumped to get things rolling and ready to cook up an awesome experience for EuroPython 2024 🚀
🇨🇿EuroPython 2024 - Prague Congress Centre 08-14 July
We are coming back to Prague! We loved last year so much that we decided to have EuroPython 2024 at the Prague Congress Centre again between the 8th and 14th of July, 2024. To stay up to date with the latest news, please visit our website.
We also encourage you to sign up to our monthly community newsletter.
Call for Proposals 🐍
The call for proposals for EuroPython 2024 is OPEN! 🎉
This year we’re aiming for a similar schedule as last year, i.e. around 120 talks, more or less 16 tutorials, posters and special events.
Tutorials/Workshops (8-9 July): will be hands-on 180 min sessions with 40-100 participants
Talks (10-12 July) : 30 or 45 min length presentations of specific and general interest to the European Python community (including a dedicated PyData track).
We are looking for sessions of all levels: beginner, intermediate, and advanced.
You are also welcome to submit posters! They will be printed in large formats. This is a graphical way to showcase your research, project or technology. Posters are exhibited at the conference, can be read at any time by participants, and can be discussed face-to-face with their authors during the Poster Sessions (between July 10th and 12th).
No matter your level of Python or public speaking experience, EuroPython's job is to help you bring yourself to our community so we all flourish and benefit from each other's experience and contribution. 🏆
The deadline for submission is March 6th, 2024 Anywhere on Earth.
We would love to see your proposals bubbling in there. Help us make 2024 the year of another incredible EuroPython!
The Speaker's Mentorship Programme is back for EuroPython 2024!
We had a call for mentees and mentors last month and it was a huge success 🚀So far, we have managed to match 28 speakers to diligent mentors.
On the 26th of February, we hosted an Ask me Anything session for all the people interested in submitting a proposal. Our programme team was there and answered all the questions raised by interested speakers. We clarified information about the Call for Proposals and shared knowledge about Financial Aid, amongst other info.
If you are curious and would like to see what was discussed, the session recording can be found on YouTube at: https://youtu.be/EiCfmf6QVIA
Call for Contributors
Conference Organisers 💪
To the wonderful humans who signed up to help organise the EuroPython Conference itself, thank you! We are late in replying to everyone and we are thankful for your patience. We got caught between dengue fevers 🦟 and carnivals 🎊.
The good news is that we are now back on track and will reach out to everyone this week.
We had a lot of applications this year, so a selection process was required. But fear nothing! In case you don’t get to work with the most fun team of all time, there is always next year. 🌟
We look forward to working together to make this conference an unforgettable experience for everyone involved 💼✨
Speaker’s mentors 🎏
For everyone who signed up to either mentor on the Speaker’s Mentorship Programme, a HUGE thank you! This would not be possible without your collaboration and work. We are so very proud to have you around learning and sharing and strengthening the community.
We truly hope you take advantage of the opportunity and develop the skills you’re looking for. After all, we believe the best way to learn is to teach. 📚✨
Proposal Reviewers 📔
We also announced the Call for Proposal Reviewers. These awesome volunteers are the people who review the proposal’s submissions and vote on the ones which they think are more suitable for making up the conference's programme.
This is very crucial work as it plays a significant part in shaping the EuroPython 2024 schedule!
A massive thank you to everyone who has filled the form to help. You will get a cucumber lollipop as a sign of our affection. 🍭
We will contact you by the end of the Call for Proposals to coordinate all the fun stuff. Please, keep an eye out for our email this month so we can start the labour. ⚒️
Fáilte Ireland Award 🏆
The 2022 edition of the EuroPython Conference (EPC) was in Dublin, Ireland. More old news? You bet! But this time it is relevant because in November 2023, the 21st edition of the EPC was awarded a Conference Ambassador Recognition Award from Fáilte Ireland, the National Tourism Development Authority for the country.
The evening was fantastic and a delightful way to acknowledge the people and businesses who have hosted conference events on the little Emerald Island. This is our first prize. We’re feeling proud and happy and wanted to share the news! 🥇
One of our members wrote more about the experience on LinkedIn.
TL;DR: We hope to do a fine job this year and help the European Python community thrive even more!
If you want to talk to us about anything, give feedback, or ask for a partner to count the stars beside you, shoot us an email at board@europython.eu.
Upcoming Events in Europe
Here are some Python events happening in Europe in the near future.
PyCon Slovakia: March 15-17 2024
PyCon SK will happen in the wonderful city of Bratislava! The conference is organised by the civic association SPy, which focuses on the promotion and dissemination of the Python programming language and other open-source technologies and ideas.
Immerse yourself in three days of Python and PyData excellence at our community-driven conference in the vibrant heart of Berlin. From workshops to live keynote sessions, connect with fellow enthusiasts and experts alike! Join fellow Pythonistas at the iconic BCC venue at Alexanderplatz where we actually hosted EuroPython 2014! Check out their website for more info: https://2024.pycon.de/
PyCon Italy: May 22-25 2024
PyCon Italia 2024 will happen in Florence. The birthplace of Renaissance will receive a wave of Pythonistas looking to geek out this year. The schedule is online and you can check it out at their cutely animated website ( https://2024.pycon.it/). Beginners will have a special treat by having a full day of activities on May 22nd. Starting with Workshops about Python, Data Science, and Public Speaking.
It’s time to get tickets!
GeoPython: May 27-29 2024
GeoPython 2024 will happen in Basel, Switzerland. Focused on exploring the fascinating fusion of Python programming and the boundless world of Geo, the 9th edition of GeoPython has already published a preliminary schedule (here: https://2024.geopython.net/schedule).
DjangoCon Europe 2024 will happen in Vigo, Spain. Organized by Django practitioners from all levels, the 16th edition of the Conference will be hosted in the beautiful Galician city, famous for its amazing food & the Illas Atlanticas.
You can check more information about Django Con Europe at their lovely website: https://2024.djangocon.eu/
Py.Jokes
pip install pyjokesimport
pyjokesprint(pyjokes.get_joke())
Child: Dad, why does the Sun rise in the East and set in the West?
Dad: Son, it's working, don't touch it!
That's all, folks! 🏁
Thanks for reading along.
We will have a regular monthly appearance in your inbox from now on. We are happy to tailor our future editions to accommodate what you'd like to see here. Influence what gets to your inbox by dropping us a line at communications@europython.eu
Excitement is in the air as we gear up for EuroPython 2024, the conference that brings together the brightest minds in the European Python community!
In the spirit of inclusivity and community collaboration, we are thrilled to announce an extension of the Call for Contributors for EuroPython 2024! We want to ensure that everyone who wishes to contribute has the opportunity to do so. This extension allows more Pythonistas to be a part of this incredible event, shaping the conference and making it an enriching experience for all.
How Can You Get Involved?
1. Review the proposals:
Have you navigated the ins and outs of proposal submissions? The heart of EuroPython lies in its program. If you're passionate about curating an exceptional conference agenda, we welcome you to join our team of Reviewers. By reviewing proposals and assisting in the selection process, you'll contribute to crafting a lineup that reflects the diversity and innovation within the Python community.
2. Volunteer for the Conference Organisation:
EuroPython is not just about talks; it's about fostering connections and creating memorable experiences. We're looking for volunteers to help with various aspects of the conference, from registration to technical support. Your contribution, big or small, will play a crucial role in making EuroPython 2024 an unforgettable event for everyone involved.
3. Become a Speaker Mentor:
Do you have experience as a speaker at Python conferences? Are you passionate about mentorship? We invite you to be a guiding force for our aspiring speakers. By volunteering as a Speaker Mentor, you'll play a pivotal role in shaping the future of Python conference speakers. Your expertise will help potential speakers refine their proposals, ensuring a diverse and high-quality program.
Provide details about your experience and areas of expertise.
Our team will match you with potential speakers based on your background.
📝 Make sure to sign up by February 12th 2024 AoE! 📝
Why Volunteer?
Community Contribution:
Contribute to the Python community by sharing your knowledge and experience.
Help newcomers find their voice in the tech world.
Networking Opportunities:
Connect with like-minded individuals, speakers, and fellow volunteers.
Build relationships that can shape your professional journey.
Personal Growth:
Hone your mentoring and organisational skills.
Gain insights into the latest trends and innovations in the Python ecosystem.
Don't miss this chance to be an integral part of EuroPython 2024! Let's join forces to make this conference a celebration of Python's diversity, innovation, and community spirit. Together, we can create an event that inspires, educates, and leaves a lasting impact on every participant.
We’re delighted to announce that EuroPython 2024 will be held again in Prague between the 8th and 14th of July, 2024. To stay up to date with the latest news please visit our website and we encourage you to sign up to our monthly community newsletter.
Sitting on the Vltava river, the city boasts numerous cultural, culinary, architectural, artistic, civic and tourist attractions. The beautiful historic centre is a UNESCO world heritage site. So, in addition to the conference, there's a LOT to see, do and enjoy. Prague has an extensive and modern public transport system, so travelling around won't be a problem and getting there is easy via well serviced train, air and road infrastructure. With an average July high of 25°c, remember to pack your shorts and sandals with your laptop, along with a spirit of adventure to explore one of the most visited and vibrant cities in Europe.
As with previous years, Monday and Tuesday (8th and 9th) will be for tutorials and workshops; the main conference talks will take place on Wednesday, Thursday and Friday (10th, 11th and 12th) with Saturday and Sunday (13th and 14th) for community sprints.
Our prospectus for sponsorship will be forthcoming soon, but if you already want to sponsor one of Europe’s biggest, friendliest and longest running community organised software development conferences, please do reach out to us at sponsoring@europython.eu and we’d be delighted to help.
Because EuroPython is a community organised conference, our volunteers are at the heart of everything we do. Without them, there simply wouldn’t be a conference. We’ll soon start recruiting volunteers, who are organised into teams responsible for different aspects of the conference. To register your interest please email volunteers@europython.eu.
Hello there and welcome to the post conference newsletter! We really hope you enjoyed EuroPython 2023 cause we sure did and are still recovering from all the fun and excitement. 😊
We have some updates to share with you and also wanted to use this newsletter to nostalgically look back at all the good times 🙌 we had in Prague just a month ago. Surrounded by old friends and new in the beautiful city of Prague, EuroPython 2023 was special for a lot of us 🤗 and the community, so we want to highlight some of those experiences!! So without further ado let’s get into the updates 🐍
EuroPython Society
The EPS board is working with our accountant and auditor to get our financial reports in order in the next couple of weeks. As soon as that is finalised, we will be excited to call for the next Annual General Assembly (GA); the actual GA will be held at least 14 days after our formal notice.
General Assembly is a great opportunity to hear about EuroPython Society's developments and updates in the last year & a new board will also be elected at the end of the GA.
The EPS board is made up of up to 9 directors (including 1 chair and 1 vice chair); the board runs the day-to-day business of the EuroPython Society, including running the EuroPython conference series, and supports the community through various initiatives such as our grants programme. The board collectively takes up the fiscal and legal responsibility of the Society.
At the moment, running the annual EuroPython conference is a major task for the EPS. As such, the board members are expected to invest significant time and effort towards overseeing the smooth execution of the conference, ranging from venue selection, contract negotiations, and budgeting, to volunteer management. Every board member has the duty to support one or more EuroPython teams to facilitate decision-making and knowledge transfer.
In addition, the Society prioritises building a close relationship with local communities. Board members should not only be passionate about the Python community but have a high-level vision and plan for how the EPS could best serve the community.
How can you become an EPS 2024 board member?
Any EPS member can nominate themselves for the EPS 2024 board. Nominations will be published prior to the GA.
Though the formal deadline for self-nomination is at the GA, it is recommended that you send in yours as early as possible (yes, now is a good time!) to board@europython.eu.
We look forward to your email :)
& for more information check out our Call for Board Candidates!
Conference Numbers
With 142 Talks, 22 Tutorials, 10 Special events, 5 Keynotes, 3 panel discussions happening throughout the week, our “learn more about this” bookmarks list/backlog reached new heights this year! If you know what I mean 😉
Let's take a closer look at our stats, if you too are into that kinda thing.
Thank you Volunteers & Sponsors <3
Year after year EuroPython shines because of the hard work of our amazing team of volunteers
But beyond the logistics and the schedules, it's your smiles, your enthusiasm, and your genuine willingness to go the extra mile that truly made EuroPython 2023 truly special. Your efforts have not only fostered a sense of belonging among first time attendees but also exemplified the power of community and collaboration that lies at the heart of this conference.
Once again, thank you for being the backbone of EuroPython, for your dedication, and for showing the world yet again why people who come for the Python language end up staying for the amazing community :)
We know how much you would love to see and share videos of some amazing talks and keynotes we had during the conference. Rest assured we are working with our AV team to have videos edited and ready in a month or so. Stay tuned for that.
In the meantime if you want to revisit a talk you missed or just want to check out a talk again, all the live streams from across the conference days is still available on our page
We also have some really sweet highlight videos featuring the amazing humans of EuroPython! Check it out on Youtube.
Community write-ups
It warms our hearts to see posts from the community about their experience and stories this year! Here are some of them, please feel free to share yours by tagging us on socials @europython or mailing us at news@europython.eu
I wanted to express my sincere gratitude for providing me with the opportunity to attend EuroPython 2023 remotely and free of charge. The conference was truly exceptional! The speakers were incredible, and their presentations were both informative and inspiring. I learned so much from each session. This being my first-ever conference experience, I never imagined it would be so captivating and enlightening. Moreover, I was particularly impressed by the sense of community that was evident throughout the event. Once again, thank you for this incredible opportunity. I am truly grateful for the experience, and if the chance arises, I would be delighted to attend future events organized by EuroPython.
Messages like these warm our hearts and pushes us to do better for the 🐍 community every single year ❤️
Code of Conduct
Code of Conduct Transparency Report is now published on our website
As EuroPython 2023 has concluded we would like to extend a heartfelt thanks to the community and our sponsors for their support throughout the event! We would like to take this opportunity to share with you some exciting job profiles from our sponsors.
Optiver
As one of the oldest market making firms in the world, Optiver has been improving financial markets since 1986. By providing liquidity to markets across the globe, we make markets more efficient, transparent and stable.
Powered by technological innovation, we continuously provide competitive, two-sided prices for thousands of financial instruments on all major exchanges globally.
Our world class teams of highly skilled professionals come from a broad range of disciplines but are bound by one simple purpose: improving the market.
Numberly helps its clients collect, analyse and leverage their data across all channels to make marketing more relevant, effective and measurable. We are more than 200 engineers divided into teams with a human dimension and freedom of speech that enables everyone to have an impact and participate in the Group's development. Our sustained growth encourages us to constantly question our technical and organisational choices.
Our challenges are international: 8 offices worldwide, and more than 30 nationalities in our main office in Paris.
Our wide range of interconnected products and technical challenges are diverse and sophisticated. Our daily missions consist in processing thousands of requests per second, operating databases of several petabytes, automating our entire bare-metal infrastructure, and building the digital marketing interfaces of tomorrow.
Data Engineer / DataOps - join teams dedicated to Big Data and Event-Driven data pipelines - design, build, operate and maintain large data storage architectures using Hadoop, Spark, Airflow, ScyllaDB, Kafka - interact with other teams to proactively solve scaling problems (Data Scientists & Software Engineers…) - build cutting edge use-case in different kind of industries You can find more details here: https://nubr.ly/DE
Temporal helps developers focus on the important things, like writing business logic, and spend less time worrying about the underlying infrastructure. We have a great open-source community filled with awesome developers (who are also our users!) providing us with real-time feedback and contributions. Temporal is backed by top VC firms, with a well built team of professionals from various successful start-ups and well-known technology companies.
Developer Relations Engineer: The Developer Relations Engineer will be Temporal’s strongest connection into the community, through their code demos utilising Temporal and adjacent technologies, sharing that story through written content and verbalising through talks. You will be an integral part of the success of Temporal’s customer journey and life cycle. Your biggest allies will be in Developer Education and Engineering, while partnering with Marketing on how the story evolves. Apply here: https://jobs.lever.co/temporal/ff1b33fe-fd12-4d8c-a266-4b1a83295182
Sentry
Bad software is everywhere, and we’re tired of it. Sentry is on a mission to help developers write better software faster, so we can get back to enjoying technology.
With more than $217 million in funding and 85,000 organizations that believe we’re on to something, we're building performance and error monitoring tools that help companies like Disney, Microsoft, and Atlassian spend less time fixing bugs and more time building products. If you like to selfishly build things that make your digital life better, come help us build the next generation of software monitoring tools.
At Picnic, we’ve been revolutionizing the way people buy their groceries since 2015 – and we’ve come a long way since then. We provide our rapidly-growing customer base with an affordable and sustainable service through cutting-edge technology, efficient planning, and a fleet of electric vehicles.
One of the main things that makes Picnic different is that we do almost everything in-house. Forecasting. Warehouse systems. Partnerships. Fleet strategies. Internal and external applications. We find it gives us more freedom when it comes to trying out new and innovative ideas!
Kiwi.com uncovers cheap travel options that others simply can't find.
Senior Python Engineer for the FinTech tribe We deal with all the monetary flows in Kiwi.com. More precisely, we are a Tribe of developers, payment specialists, business & data analysts and product magicians inside Kiwi.com in charge of building the pillars of everything that is directly or indirectly related to payments in Kiwi.com. Payments from our customers, payments to airlines and travel agencies, preventing fraudulent , anything you can imagine in that matter, we’re involved.
We also have a whole bunch of digital swag from our sponsors!🐍 💝 They will only be available for a limited time so make sure to check them out on https://ep2023.europython.eu/virtual-swag
Learn how our tool can help your web development, data science, or machine learning work get done.
Whether you’re a web developer, a data scientist, or a machine learning engineer, you want to get your work done fast and achieve the best possible results. This is exactly where your choice of tooling comes in.
Having everything set up for your work, getting smart assistance and code insight, having a seamless integration with your databases and Git repositories can make a huge difference, both keeping you in the loop and creating extra inspiration.
This is what PyCharm is built for. To help you get your work done. In the most productive way.
PyCharm is an integrated development environment designed for all Python workflows, including full-stack web development, data science, and machine learning engineering.
PyCharm allows you to set your project environment straight out of the box, offering integration with your databases, Git, and other development tools – all without leaving the IDE.
With PyCharm, you can be confident to navigate and refactor large codebases with ease, and even debug complex code sets.
AI assistant in PyCharm 2023.2
The upcoming release will bring a new AI Assistant tool to PyCharm. Let’s take a closer look at what this new feature has got in store.
AI chat
We’ve implemented an AI Assistant tool window that you can use to have a conversation with the LLM, ask questions, or iterate on a task. The IDE will provide some project-specific context, such as the languages and technologies used in your project. You will be able to ask the AI about a specific code fragment, with functionality such as Explain code, Suggest refactoring, and Find potential problems.
Documentation generation
You will be able to generate the documentation for a declaration using an LLM.
Name suggestions
When you rename a Python declaration, the AI will suggest name ideas variants for the declaration, based on its contents.
Commit message generation
The commit message dialog will have a Generate Commit Message with AI Assistant button. Clicking it you will be able to send the diffs of your changes to the LLM, which will generate a commit message describing your changes.
You can learn more about AI Assistant in PyCharm in this blog post.
Never used PyCharm before? Here are some of the features our team recommends!
Mukul Mantosh, PyCharm Content Writer, on his favorite features for Django development:
In 2017, I made the decision to transition from PHP to Python, embarking on a journey to learn the language. During my exploration, I stumbled upon the Django framework. At the time, I was on the lookout for an IDE that would cater to my specific needs. I experimented with various options, but none of them seemed to fulfill all my requirements. That is, until I discovered PyCharm. Ever since then, I haven't looked back in nearly five years.
PyCharm offers exceptional support for Django, particularly through features like Live Templates, Run/Debug Configuration, and Python Packaging. One of the standout features is the ability to check for the latest version of packages, such as NumPy, without the need to visit external websites like pypi.org. This functionality can be accessed directly from within the IDE. The design of PyCharm is meticulously crafted to keep everything conveniently in one place, whether it's integrating with version control systems (VCS), utilizing DataGrip for databases, or collaborating with other remote developers (via Code with Me). Personally, one of my favorite aspects is the flexibility of running applications either through a remote interpreter on a cloud server or via the Windows Subsystem for Linux (WSL) on a Windows machine.
Jodie Burchell, JetBrains developer advocate for data science, on the functionality that helps her efficiently work on her data science projects:
Coming into data science from an academic background, I didn’t think much about my tooling. In fact, I was happy with plain Jupyter notebooks without any extensions for many years! When I started using PyCharm for my data science and machine learning work, I was amazed at how much easier it was to work with both Jupyter notebooks and large code bases. Even core PyCharm features such as setting up a new virtual or Conda environment in a few clicks, code completion, inbuilt documentation, and code introspection made me much more productive and saved me a lot of time, and made me wonder why I hadn’t tried to update my tooling earlier.
Over time, I started exploring more advanced features, such as the ability to connect to and query databases in SQL right within the IDE and the UI to help me fix my *many* mistakes with Git. PyCharm also saved me when I needed to understand and document a large legacy codebase for a machine learning product. As the entire codebase was indexed when I opened it in PyCharm, it was simple for me to find all uses of methods and piece together what was happening.
Learn more from PyCharm team at Europython
Come to our booth!
If you want to learn how you can get your work done faster and stay in the flow longer, come to the JetBrains booth to talk to members of the PyCharm team, such as our developer advocates Paul Everitt, Jodie Burchell, and Helen Scott.
Join a talk “Power Django with PyCharm”
Join a talk by Paul Everitt to learn how your IDE can make your development joyful. Paul will also demonstrate “test-first” development within the IDE, plus how this can apply to frontends. There will be a surprise gift at the end! Catch up with Paul at the PyCharm booth to learn more about full-stack web development and test driven development with Pytest.
Challenge your knowledge and win a prize
For those of you who are up for a challenge, we will be having two web development and two machine learning quizzes a day at our booth from Wednesday through to Friday. Check the quiz schedule at the booth. The top three scorers in each web development quiz will win a copy of Boost Your Django DX by Adam Johnson, while the top three winners in each machine learning quiz will have their choice of Matt Harrison’s Effective Pandas or Effective XGBoost. Best of luck!
We’re thrilled to be a Platinum Sponsor of EuroPython again this year, happening from July 17th-23rd in Prague, Czechia. If you can’t make it in person, you can still attend the conference remotely, as EuroPython is a hybrid event this year!
Our team member: Steve Dower, will be giving talks during the conference:
Make sure you don’t miss them! We’ll also be at the Microsoft booth talking about the hard work our teams have been doing to continue supporting the Python community, and to improve the experience for Python developers across our products over the past year.
We have also improved Data Science workflows inside of Visual Studio Code. Polyglot notebooks now supports Python, allowing you to use multiple programming languages natively all in the same notebook in Visual Studio Code! No more needing wrapper libraries or magic commands to work with your favorite languages in the same notebook! With polyglot notebooks, each language in Polyglot Notebooks gets a first-class editing experience with language server support such as autocompletion, syntax highlighting, and signature help. Variable explorer now allows you to check values and share variables across all the supported languages.
Additionally, we have launched the Data Wrangler extension in Visual Studio Code. With Data Wrangler, you can seamlessly clean and explore your data in VS Code. It offers a variety of features that will help you quickly identify and fix errors, inconsistencies, and missing data. You can perform data profiling and data quality checks, visualize data distributions, and easily transform data into the format you need. Plus, Data Wrangler comes with a library of built-in transformations and visualizations, so you can focus on your data, not the code. As you make changes, the tool generates code using open-source Python libraries for the data transformation operations you perform. This means you can write better data preparation programs faster and with fewer errors. The code also keeps Data Wrangler transparent and helps you verify the correctness of the operation as you go.
In the machine learning and AI realm, Azure Machine Learning Visual Studio Code Web integration is now in public preview. VS Code for the Web provides you with a full-featured development environment for building your machine learning projects, all from the browser and without required installations or dependencies. And by connecting your Azure Machine Learning compute instance, you get the rich and integrated development experience VS Code offers, enhanced by the power of Azure Machine Learning. Furthermore, Prompt Flow for Azure Machine Learning is now in preview! Prompt Flow provides a streamlined experience for prompting, evaluating, tuning, and operationalizing large language models. With prompt flow, you can quickly create prompt workflows that connect to various language models and data sources. This allows for building intelligent applications and assessing the quality of your workflows to choose the best prompt for your case.
These are only some of the many things we look forward to chatting with you at EuroPython. But if you can’t make it there, no problem - you can always connect to us through our Discord channel.
Whether it’s in person or virtually, we all look forward to meeting you!
Hi, I’m David, a Python developer at Kraken Technologies. I work on Kraken: a Python application which has, at last count, 27,637 modules. Yes, you read that right: nearly 28k separate Python files - not including tests. I do this along with 400 other developers worldwide, constantly merging in code. And all anyone needs to make a change - and kick start a deployment of the software that runs 17 different energy and utility companies, with many millions of customers - is one single approval from a colleague on Github.
Now you may be thinking this sounds like a recipe for chaos. Honestly, I would have said the same. But it turns out that large numbers of developers can, at least in the domain we work in, work effectively on a large Python monolith. There are lots of reasons why this is possible, many of them cultural rather than technical, but in this blog post I want to explain about how the organisation of our code helps to make this possible.
Layering our code base
If you’ve worked on a code base for any length of time, you will have felt the drift towards unpleasant complexity. Strands of logic tangle together across your application, and it becomes increasingly difficult to think about parts of it in isolation. This is what started happening to our young code base, and so we decided to adopt what is known as a ‘layered architecture’ where there are constraints about what parts of the code base can know about each other.
Layering is a well-known software architecture pattern in which components are organized, conceptually, into a stack. A component is not allowed to depend on any components higher up the stack.
For example, in the above diagram, C would be allowed to depend on B and A, but not D.
The idea of a layered architecture is broad: it may be applied to different kinds of components. For example, you could layer several independently-deployable services; or alternatively your components could just be a set of source code files.
What constitutes a dependency is also broad. In general, if a component has direct knowledge of another component (even if purely at a conceptual level) then it depends on it. Indirect interaction (e.g. via configuration) is not usually seen as a dependency.
Layers in Python
In a Python code base, the layers are best thought of as Python modules, and dependencies as import statements.
The top-level modules and subpackages are good candidates for layers. Let’s say we decide the layers should be in this order:
shopping_cart
payments
products
Our architecture would thus forbid, for example, any of the modules within payments from importing from shopping_cart. They could, however, import from products.
Layering can also be nested, so we could choose to layer within our payments module like so:
api
vendor
There’s no single, correct way of choosing which layers exist, and in which order - that’s an act of design. But layering like this leads to a less tangled code base, making it easier to understand and change.
How we’ve layered Kraken
At the time of writing, 17 different energy and utility companies license Kraken. We call these companies clients, and run a separate instance for each. Now, one of Kraken’s main characteristics is that different instances are ‘the same, but different’. In other words, there is a lot of shared behavior, but also every client has bespoke code that defines their specific needs. This is also true at the territory level: there are commonalities between all the clients that run in Britain (they integrate with the same energy industry) that aren’t shared with, say, Octopus Energy Japan.
As Kraken grew into a multi-client platform, we evolved our layering to help with this. Broadly speaking, it now looks like this at the top level:
The clients layer is at the top. Each client gets a subpackage inside that layer (for example, oede corresponds to Octopus Energy Germany). Below that is territories, for all the country-specific behaviour, again with territory-specific subpackages. The bottom layer is core, which contains code that is used by all clients. There is an additional rule, which is that client subpackages must be independent (i.e. not import from other clients), and the same goes for territories.
Layering Kraken like this allows us to make changes with a limited ‘blast radius’. Because the clients layer is at the top, nothing depends on it directly, making it easier to change something that relates to a particular client without accidentally affecting behavior on a different client. Likewise, changes that relate only to one territory won’t affect anything in a different one. This allows us to move quickly and independently across teams, especially when we are making changes that only affect a small number of Kraken instances.
Enforcing layering with Import Linter
When we introduced layering, we quickly found that just talking about the layering was not enough. Developers would often accidentally introduce layering violations. We needed to enforce it somehow, and we do this using Import Linter.
Import Linter is an open source tool for checking that you are following layered architectures. First, in an INI file you define a contract describing your layering - something like this
[importlinter:contract:top-level]
name = Top level layers
type = layers
layers =
kraken.clients
kraken.territories
Kraken.core
We can also enforce the independence of the different clients and territories, using two more contracts (this time `independence` contracts)
[importlinter:contract:client-independence]
name = Client independence
type = independence
layers =
kraken.clients.oede
kraken.clients.oegb
kraken.clients.oejp
...
[importlinter:contract:territory-independence]
name = Territory independence
type = independence
layers =
kraken.territories.deu
kraken.territories.gbr
kraken.territories.jpn
...
Then you can run lint-imports on the command line and it will tell you whether or not there are any imports that break our contracts. We run this in the automated checks on every pull request, so if someone introduces an illegal import, the checks will fail and they won’t be able to merge it.
These are not the only contracts. Teams can add their own layering deeper in the application: kraken.territories.jpn, for example, is itself layered. We currently have over 40 contracts in place.
Burning down technical debt
When we introduced the layered architecture, we weren’t able to adhere to it from day one. So we used a feature in Import Linter which allows you to ignore certain imports before checking the contract.
[importlinter:contract:my-layers-contract]
name = My contract
type = layers
layers =
kraken.clients
kraken.territories
kraken.core
ignore_imports =
kraken.core.customers ->
kraken.territories.gbr.customers.views
kraken.territories.jpn.payments -> kraken.utils.urls
(and so on...)
We then used the number of ignored imports as a metric for tracking technical debt. This allowed us to observe whether things were improving, and at what rate.
Here’s our graph of how we’ve been working through ignored imports over the last year or so. Periodically I share this to show people how we’re doing and encourage them to work towards complete adherence. We use this burndown approach for several other technical debt metrics too.
Downsides, there are always downsides
Local complexity
At some point after adopting a layered architecture, you will run into a situation where you want to break the layers. Real life is complex, there are interdependencies everywhere, and you will find yourself wanting to, say, call a function that’s in a higher layer.
Fortunately, there is always a way around this. It’s called inversion of control and it’s easy to do in Python, it just requires a mindset shift. But it does lead to an increase in ‘local’ complexity (i.e. in a little part of your code base). However, it’s a price worth paying for a simpler system overall.
Too much code in higher layers
The higher the layer, the easier the change. We deliberately made it easy to change code for specific clients or territories. Code in the core, which everything depends on, is more costly and risky to make changes to.
As a result, there has been a design pressure, brought about partly by the layering we chose, to write more client and territory-specific rather than introduce deeper, more globally useful code into the core. As a result, there is more code in the higher layers than we might ideally like. We’re still learning about how to tackle this.
We’re still not finished
Remember those ignored imports? Well, years on, we still have some! At last count, 15. Those last few imports are the stubbornest, most tangled ones of all.
It can take serious effort to retrospectively layer a code base. But the sooner you do it, the less tangling you’ll have to address.
In summary
Layering Kraken has kept our very large code base healthy and relatively easy to work with, especially considering its size. Without imposing constraints on the relationships between the tens of thousands of modules, our code base would probably have tangled into an enormous plate of spaghetti. But the large scale structure we chose - and evolved along with the business - has helped us work in large numbers on a single Python code base. It shouldn’t be possible, but it is!
If you’re working on a large Python codebase - or even a relatively small one - give layering a try. The sooner you do, the easier it will be.
Kraken Technologies LTD's is sponsor of EuroPython 2023, check them out on https://kraken.tech/
Among the many wonderful workshops at EuroPython this year, we are pleased to announce we will be running the Humble Data workshop in person on Monday 17th July 2023, at the Prague Congress Centre (PCC). This is following successful deliveries of this workshop at PyCon US, Ghana, Namibia, Africa, PyData Global and of course, EuroPython 2022!
How is the workshop?
Curious about the event? Read on.
Humble Data workshops are designed to get those from underrepresented groups started in both Python and data science, in an inclusive, laid-back and empathic environment. The workshops focus on helping people with zero experience with coding to learn some of the most fundamental operations in Python, and in turn, use these to get started with reading, transforming and visualizing data.
The workshop is structured to last for 3 hours.
As part of the workshop, participants will work through a series of approachable tutorials with the help of a mentor. For 3 hours (breaks included) we will have teams that will work together with mentors to do plenty of exercises, quizzes and games, to go from Zero to Hero in Python and data science. All that participants will need to bring is a tablet or a laptop with internet access - we will help them with the rest!
Let us help you get started on your Python data science journey. You can read more about the workshop here.
How can I get involved?
Like the idea? Join us as a mentor or mentee!
If you’re new to coding or data science and want to learn more in a supportive environment, apply to join us at the Humble Data workshop by filling in this form. Participation is free for anyone with a EuroPython Conference Ticket or Combined Ticket.
If you’re interested in mentoring, we would love to have your help! It is no issue if you're not the most experienced programmer or data scientist: rather, we are looking for people who are respectful, patient, friendly, curious, and able to explain technical concepts in a way that is approachable for beginners. In return, you will receive the eternal gratitude of the organizers and attendees, the chance to meet people outside of your bubble, and in turn, show that you don’t need to fit a specific mould to “look like” a developer or data scientist.
Finally, if you know anyone attending EuroPython this year who you think would like to either attend or mentor Humble Data, please encourage them to apply!
If you’re interested in attending or mentoring, please complete this form by July 3rd, 2023.