Python Bytes

by Michael Kennedy and Brian Okken

Python Bytes is a weekly podcast hosted by Michael Kennedy and Brian Okken. The show is a short discussion on the headlines and noteworthy news in the Python, developer, and data science space.

  

Latest Episodes

#109 CPython byte code explorer

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: Python Descriptors Are Magical Creatures

  • an excellent discussion of understanding @property and Python’s descriptor protocol.
  • discussion includes getter, setter, and deleter methods you can override.

Michael #2: Data Science Survey 2018 JetBrains

  • JetBrains polled over 1,600 people involved in Data Science and based in the US, Europe, Japan, and China, in order to gain insight into how this industry sector is evolving
  • Key Takeaways
    • Most people assume that Python will remain the primary programming language in the field for the next 5 years.
    • Python is currently the most popular language among data scientists.
    • Data Science professionals tend to use Keras and Tableau, while amateur data scientists are more likely to prefer Microsoft Azure ML.
  • Most common activities among pros and amateurs:
    • Data processing
    • Data visualization
  • Main programming language for data analysis
    • Python 57%
    • R 15%
    • Julia 0%
  • IDEs and Editors
    • Jupyter 43%
    • PyCharm 38%
    • RStudio 23%

Brian #3: cache.py

  • cache.py is a one file python library that extends memoization across runs using a cache file.
  • memoization is an incredibly useful technique that many self taught or on the job taught developers don’t know about, because it’s not obvious.
  • example:
    import cache

    @cache.cache()
    def expensive_func(arg, kwarg=None):
      # Expensive stuff here
      return arg
  • The @cache.cache() function can take multiple arguments.
    • @cache.cache(timeout=20) - Only caches the function for 20 seconds.
    • @cache.cache(fname="my_cache.pkl") - Saves cache to a custom filename (defaults to hidden file .cache.pkl)
    • @cache.cache(key=cache.ARGS[KWARGS,NONE]) - Check against args, kwargs or neither of them when doing a cache lookup.

Michael #4: Setting up the data science tools

  • part of a larger video series
  • set up. Tools to keras ultimately
  • Tools
    • anaconda
    • tensorflow
    • Jupyter
    • Keras
  • good for true beginners
  • setup and activate a condo venv
  • Start up a notebook and switch envs
  • use conda, rather than pip

Brian #5: chartify

  • “Python library that makes it easy for data scientists to create charts.”
  • from the docs:
    • Consistent input data format: Spend less time transforming data to get your charts to work. All plotting functions use a consistent tidy input data format.
    • Smart default styles: Create pretty charts with very little customization required.
    • Simple API: We've attempted to make to the API as intuitive and easy to learn as possible.
    • Flexibility: Chartify is built on top of Bokeh, so if you do need more control you can always fall back on Bokeh's API.

Michael #6: CPython byte code explorer

  • JupyterLab extension to inspect Python Bytecode
  • via Anton Helm
  • by Jeremy Tuloup
  • You’ll see exactly what it’s about if you watch the GIF movie at the github repo.
  • Can’t think of a better way to understand Python bytecode quickly than to play a little with this
  • Comparing versions of CPython: If you have several versions of Python installed on your machine (let's say in different conda environments), you can use the extension to check how the bytecode might differ.
  • Nice visualization of different performance aspects of while vs. for at the end

Extras:

Brian:


Audio Download

Posted on 18 December 2018 | 8:00 am


#108 Spilled data? Call the PyJanitor

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: pyjanitor - for cleaning data
  • originally a port of an R package called janitor, now much more.
  • “pyjanitor’s etymology has a two-fold relationship to “cleanliness”. Firstly, it’s about extending Pandas with convenient data cleaning routines. Secondly, it’s about providing a cleaner, method-chaining, verb-based API for common pandas routines.”
  • functionality:
    • Cleaning columns name (multi-indexes are possible!)
    • Removing empty rows and columns
    • Identifying duplicate entries
    • Encoding columns as categorical
    • Splitting your data into features and targets (for machine learning)
    • Adding, removing, and renaming columns
    • Coalesce multiple columns into a single column
    • Convert excel date (serial format) into a Python datetime format
    • Expand a single column that has delimited, categorical values into dummy-encoded variables
  • This pandas code:
    df = pd.DataFrame(...)  # create a pandas DataFrame somehow.
    del df['column1']  # delete a column from the dataframe.
    df = df.dropna(subset=['column2', 'column3'])  # drop rows that have empty values in column 2 and 3.
    df = df.rename({'column2': 'unicorns', 'column3': 'dragons'})  # rename column2 and column3
    df['newcolumn'] = ['iterable', 'of', 'items']  # add a new column.
- looks like this with pyjanitor:
    df = (
        pd.DataFrame(...)
        .remove_columns(['column1'])
        .dropna(subset=['column2', 'column3'])
        .rename_column('column2', 'unicorns')
        .rename_column('column3', 'dragons')
        .add_column('newcolumn', ['iterable', 'of', 'items'])
    )
Michael #2: What Does It Take To Be An Expert At Python?
  • Presentation at PyData 2017 by James Powell
  • Covers Python Data Model (dunder methods)
  • Covers uses of Metaclasses
  • All done very smoothly as a series of demos
  • Pretty long and in depth, 1.5+ hours
Brian #3: Awesome Python Applications
  • pypi is a great place to find great packages you can use as examples for the packages you write. Where do you go for application examples? Well, now you can go to Awesome Python Applications.
  • categories of applications included: internet, audio, video, graphics, games, productivity, organization, communication, education, science, CMS, ERP (enterprise resource planning), static site generators, and a whole slew of developer related applications.
  • Mahmoud is happy to have help filling this out, so if you know of a great open source application written in Python, go ahead and contribute to this, or open an issue on this project.
Michael #4: Django Core no more
  • Write up by James Bennett
  • If you’re not the sort of person who closely follows the internals of Django’s development, you might not know there’s a draft proposal to drastically change the project’s governance.
  • What’s up: Django the open-source project is OK right now, but difficulty in recruiting and retaining enough active contributors.
  • Some of the biggest open-source projects dodge this by having, effectively, corporate sponsorship of contributions.
  • Django has become sort of a victim of its own success: the types of easy bugfixes and small features that often are the path to growing new committers have mostly been done already in Django.
  • Not managed to bring in new committers at a sufficient rate to replace those who’ve become less active or even entirely inactive, and that’s not sustainable for much longer.
  • Under-attracting women contributors too
  • Governance: Some parallels to what the Python core devs are experiencing now. Project leads BDFLs stepped down.
  • The proposal: what I’ve proposed is the dissolution of “Django core”, and the revocation of almost all commit bits
    • Seems extreme but they were working much more as a team with PRs, etc anyway.
    • Breaks down the barrier to needing to be on the core team to suggest, change anything.
    • Two roles would be formalized — Mergers and Releasers — who would, respectively, merge pull requests into Django, and package/publish releases. But rather than being all-powerful decision-makers, these would be bureaucratic roles
Brian #5: wemake django template
  • a cookie-cutter template for serious django projects with lots of fun goodies
  • “This project is used to scaffold a django project structure. Just like django-admin.py startproject but better.”
  • features:
    • Always up-to-date with the help of [@dependabot](https://dependabot.com/)
    • poetry for managing dependencies
    • mypy for optional static typing
    • pytest for unit testing
    • flake8 and wemake-python-styleguide for linting
    • pre-commit hooks for consistent development
    • docker for development, testing, and production
    • sphinx for documentation
    • Gitlab CI with full build, test, and deploy pipeline configured by default
    • Caddy with https and http/2 turned on by default
Michael #6: Django Hunter
  • Tool designed to help identify incorrectly configured Django applications that are exposing sensitive information.
  • Why? March 2018: 28,165 thousand django servers are exposed on the internet, many are showing secret API keys, database passwords, amazon AWS keys.
  • Example: https://twitter.com/6IX7ine/status/978598496658960384
  • Some complained this inferred Django was insecure and said it wasn’t. Others thought “There is a reasonable argument to be made that DEBUG should default to False.”
  • One beginner, Peter, chimes in:
    • I probably have one of them, among my early projects that are on heroku and public GitHub repos.
    • I did accidentally expose my aws password this way and all hell broke loose.
    • The problem is that as a beginner, it wasn't obvious to me how to separate development and production settings and keep production stuff out of my public repository.
Extras:

Michael: Thanks for having me on your show Brian: https://blog.michaelckennedy.net/2018/12/08/being-a-great-podcast-guest/

Brian: open source extra: For Christmas, I want a dragon…

pic.twitter.com/RmFAEgqpSr — Changelog (@changelog)

Michael: Why did the multithreaded chicken cross the road?

  • road the side get to the other of to
  • to get the side to road the of other
  • the side of to the to road other get
  • to of the road to side other the get


Audio Download

Posted on 11 December 2018 | 8:00 am


#107 Restructuring and searching data, the Python way

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: glom: restructuring data, the Python way
  • glom is a new approach to working with data in Python, featuring:
    • Path-based access for nested structure
      • data\['a'\]['b']['c']glom(data, 'a.b.c')
    • Declarative data transformation using lightweight, Pythonic specifications
      • glom(target, spec, **kwargs) with options such as
        • a default value if value not found
        • allowed exceptions
    • Readable, meaningful error messages:
      • PathAccessError: could not access 'c', part 2 of Path('a', 'b', 'c') is better than
      • TypeError: 'NoneType' object is not subscriptable
    • Built-in data exploration and debugging features
      • glom.Inspect(``**a*``, ***kw*``)
      • The [**Inspect**](https://glom.readthedocs.io/en/latest/api.html#glom.Inspect) specifier type provides a way to get visibility into glom’s evaluation of a specification, enabling debugging of those tricky problems that may arise with unexpected data.
Michael #2: Scientific GUI apps with TraitsUI Brian #3: Pampy: The Pattern Matching for Python you always dreamed of
  • “Pampy is pretty small (150 lines), reasonably fast, and often makes your code more readable and hence easier to reason about.”
  • uses _ as the missing info in a pattern
  • simple match signature of match(input, pattern, action)

  • Examples

    • nested lists and tuples
    from pampy import match, _

    x = [1, [2, 3], 4]
    match(x, [1, [_, 3], _], lambda a, b: [1, [a, 3], b])           # => [1, [2, 3], 4]
  - dicts:
    pet = { 'type': 'dog', 'details': { 'age': 3 } }
    match(pet, { 'details': { 'age': _ } }, lambda age: age)        # => 3
    match(pet, { _ : { 'age': _ } },        lambda a, b: (a, b))    # => ('details', 3)
Michael #4: Google AI better than doctors at detecting breast cancer
  • Google’s deep learning AI called LYNA able to correctly identify tumorous regions in lymph nodes 99 per cent of the time.
  • We think of the impact of AI as killing 'low end' jobs [see poster], but these are "doctor" level positions.
  • The presence or absence of these ‘nodal metastases’ influence a patient’s prognosis and treatment plan, so accurate and fast detection is important.
  • In a second trial, six pathologists completed a diagnostic test with and without LYNA’s assistance. With LYNA’s help, the doctors found it ‘easier’ to detect small metastases, and on average the task took half as long.
Brian #5: 2018 Advent of Code

Another winter break activity people might enjoy is practicing with code challenges. AoC is a fun tradition.

  • a calendar of small programming puzzles for a variety of skill sets and skill levels that can be solved in any programming language you like.
  • don't need a computer science background to participate
  • don’t need a fancy computer; every problem has a solution that completes in at most 15 seconds on ten-year-old hardware.
  • There’s a leaderboard, so you can compete if you want. Or just have fun.
  • Past years available, back to 2015.
  • Some extra tools and info: awesome-advent-of-code
Michael #6: Red Hat Linux 8.0 Beta released, now (finally) updated to use Python 3.6 as default instead of 2.7
  • First of all, my favorite comment was a correction to the title: legacy python *
  • Python 3.6 is the default Python implementation in RHEL 8; limited support for Python 2.7 is provided. No version of Python is installed by default.“
    • Red Hat Enterprise Linux 8 is distributed with Python 3.6. The package is not installed by default. To install Python 3.6, use the yum install python3 command.
    • Python 2.7 is available in the python2 package. However, Python 2 will have a shorter life cycle and its aim is to facilitate smoother transition to Python 3 for customers.
    • Neither the default python package nor the unversioned /usr/bin/python executable is distributed with RHEL 8. Customers are advised to use python3 or python2 directly. Alternatively, administrators can configure the unversioned python command using the alternatives command.
  • Python scripts must specify major version in hashbangs at RPM build time
    • In RHEL 8, executable Python scripts are expected to use hashbangs (shebangs) specifying explicitly at least the major Python version.
Extras:

Michael: We were featured on TechMeme Long Ride Home podcast. Check out their podcast here. Thank you to Brian McCullough, the host of the show. I just learned about their show through this exchange but can easily see myself listening from time to time. It’s like Python Bytes, but for the wider tech world and less developer focused but still solid tech foundations.

Brian: First story was about glom. I had heard of glom before, but got excited after interviewing Mahmoud for T&C 55, where we discussed the difficulty in testing if you use glom or DSLs in general. A twitter exchange and GH issue followed the episode, with Anthony Shaw. At one point, Ant shared this great joke from Brenan Kellar:

A QA engineer walks into a bar. Orders a beer. Orders 0 beers. Orders 99999999999 beers. Orders a lizard. Orders -1 beers. Orders a ueicbksjdhd.

First real customer walks in and asks where the bathroom is. The bar bursts into flames, killing everyone.

— Brenan Keller (@brenankeller) November 30, 2018


Audio Download

Posted on 7 December 2018 | 8:00 am


#106 Fluent query APIs on Python collections

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: Dependency Management through a DevOps Lens
  • Python Application Dependency Management in 2018 - Hynek
  • An opinionated comparison of one use case and pipenv, poetry, pip-tools
  • “We have more ways to manage dependencies in Python applications than ever. But how do they fare in production? Unfortunately this topic turned out to be quite polarizing and was at the center of a lot of heated debates. This is my attempt at an opinionated review through a DevOps lens.”
  • Best disclaimer in a blog article ever:
    • DISCLAIMER: The following technical opinions are mine alone and if you use them as a weapon to attack people who try to improve the packaging situation you’re objectively a bad person. Please be nice.”
  • Requirements: Solution needs to meet the following features:
    1. Allow me specify my immediate dependencies (e.g. Django),
    2. resolve the dependency tree and lock all of them with their versions and ideally hashes (more on hashes),
    3. integrate somehow with tox so I can run my tests,
    4. and finally allow me to install a project with all its locked dependencies into a virtual environment of my choosing.
  • Seem like reasonable wishes. So far, none of the solutions work perfectly.
  • A good example of pointing out tooling issues with his use case while being respectful of the people involved in creating other tools.
Michael #2: Plugins made simple with pluginlib
  • makes creating plugins for Python very simple
  • it relies on metaclasses, but the average programmer can easily get lost dealing with metaclasses
  • Main Features:
    • Plugins are validated when they are loaded (instead of when they are used)
    • Plugins can be loaded through different mechanisms (modules, filesystem paths, entry points)
    • Multiple versions of the same plugin are supported (The newest one is used by default)
    • Plugins can be blacklisted by type, name, or version
    • Multiple plugin groups are supported so one program can use multiple sets of plugins that won't conflict
    • Plugins support conditional loading (examples: os, version, installed software, etc)
    • Once loaded, plugins can be accessed through dictionary or dot notation
Brian #3: How to Test Your Django App with Selenium and pytest
  • Bob Belderbos
  • “In this article I will show you how to test a Django app with pytest and Selenium. We will test our CodeChalleng.es platform comparing the logged out homepage vs the logged in dashboard. We will navigate the DOM matching elements and more.”
Michael #4: Fluent collection APIs (flupy and asq)
  • flupy implements a fluent interface for chaining multiple method calls as a single python expression.
  • All flupy methods return generators and are evaluated lazily in depth-first order.
  • This allows flupy expressions to transform arbitrary size data in extremely limited memory.
  • Example:
    pipeline = flu(count()).map(lambda x: x**2) \
                           .filter(lambda x: x % 517 == 0) \
                           .chunk(5) \
                           .take(3)

    for item in pipeline:
      print(item)
  • The CLI in particular has been great for our data science team. Not everyone is super comfortable with linux-fu so having a cross-platform way to leverage python knowledge on the shell has been an easy win.
  • Also if you are LINQ inclined: https://github.com/sixty-north/asq
  • asq is simple implementation of a LINQ-inspired API for Python which operates over Python iterables, including a parallel version implemented in terms of the Python standard library multiprocessing module.
    # ASQ
    >>> from asq import query
    >>> words = ["zero", "one", "two", "three", "four", "five", "six", "seven", "eight", "nine", "ten"]
    >>> query(words).order_by(len).then_by().take(5).select(str.upper).to_list()
    ['ONE', 'SIX', 'TEN', 'TWO', 'FIVE']
Brian #5: Guido blogging again
  • What to do with your computer science career
  • Answering “A question about whether to choose a 9-5 job or be an entrepreneur”
    • entrepreneurship isn’t for everyone
    • working for someone else can be very rewarding
    • shoot for “better than an entry-level web development job”
  • And “A question about whether AI would make human software developers redundant (not about what I think of the field of AI as a career choice)”
    • AI is about automating tasks that can be boring
    • Software Engineering is never boring.
Michael #6: Web apps in pure Python apps with Anvil
  • Design with our visual designer
  • Build with nothing but Python
  • Publish Instant hosting in the cloud or on-site
  • Paid product but has a free version
  • Covered on Talk Python 138
Extras:


Audio Download

Posted on 1 December 2018 | 8:00 am


#105 Colorizing and Restoring Old Images with Deep Learning

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: Colorizing and Restoring Old Images with Deep Learning
  • Text interview by Charlie Harrington of Jason Antic, developer of DeOldify
  • A whole bunch of machine learning buzzwords that I don’t understand in the slightest combine to make a really cool to to make B&W photos look freaking amazing.
  • “This is a deep learning based model. More specifically, what I've done is combined the following approaches:
    • Self-Attention Generative Adversarial Network
    • Training structure inspired by (but not the same as) Progressive Growing of GANs.
    • Two Time-Scale Update Rule.
    • Generator Loss is two parts: One is a basic Perceptual Loss (or Feature Loss) based on VGG16. The second is the loss score from the critic.”
Michael #2: PlatformIO IDE for VSCode
  • via Jason Pecor
  • PlatformIO is an open source ecosystem for IoT development
  • Cross-platform IDE and unified debugger. Remote unit testing and firmware updates
  • Built on Visual Studio Code which has a nice extension for Python
  • PlatformIO, combined with the features of VSCode provides some great improvements for project development over the standard Arduino IDE for Arduino-compatible microcontroller based solutions.
  • Some of these features are paid, but it’s a reasonable price
  • With Python becoming more popular for microcontroller design, as well, this might be a very nice option for designers.
  • And for Jason’s, specifically, it provides a single environment that can eventually be configured to handle doing the embedded code design, associated Python supporting tools mods, and HDL development.
  • The PlatformIO Core written in Python. Python 2.7 (hiss…)
  • Jason’s test drive video from Tuesday: Test Driving PlatformIO IDE for VSCode
Brian #3: Python Data Visualization 2018: Why So Many Libraries?
  • Nice overview of visualization landscape, by Anaconda team
  • Differentiating factors, API types, and emerging trends
  • Related: Drawing Data with Flask and matplotlib
    • Finally! A really simple example app in Flask that shows how to both generate and display matplotlib plots.
    • I was looking for something like this about a year ago and didn’t find it.
Michael #4: coder.com - VS Code in the cloud
  • Full Visual Studio Code, but in your browser
  • Code in the browser
  • Access up to 96 cores
  • VS Code + extensions, so all the languages and features
  • Collaborate in real time, think google docs
  • Access linux from any OS
  • Note: They sponsored an episode of Talk Python To Me, but this is not an ad here...
Brian #5: By Welcoming Women, Python’s Founder Overcomes Closed Minds In Open Source
  • Forbes’s article about Guido and the Python community actively working to get more women involved in core development as well as speaking at conferences.
  • Good lessons for other projects, and work teams, about how you cannot just passively “let people join”, you need to work to make it happen.
Michael #6: Machine Learning Basics

Extras:


Audio Download

Posted on 23 November 2018 | 8:00 am


#104 API Evolution the Right Way

Python Bytes 104

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Michael #0.1: Chapters and play at

  • Chapters are now in the mp3 file
  • Play at button on the website (doesn’t work on iOS unless you click the play to start it)

Michael #0.2: Become a friend of the show

Brian #1: wily: A Python application for tracking, reporting on timing and complexity in tests and applications.
  • Anthony Shaw (aka “Friend of the Show”, aka “Ant”)
  • (if listing 2 “aliases, do you just put one “aka” or one per alias?)
  • I should cover this on Test & Code for the content of the package. But it’s the actual packaging that I want to talk about today.
  • Wily is a code base that can be used as an example of embracing pyproject.toml (pyproject.toml discussed on PB 100 and T&C 52)
  • A real nice clean project using newer packaging tools that also has some frequently used bells and whistles
  • NO setup.py file
  • wily’s pyproject.toml includes
    • flit packaging, metadata, scripts
    • tox configuration
    • black configuration
  • project also has
    • testing done on TravisCI
    • rst based docs and readthedocs updates
    • code coverage
    • black pre-commit for wily
    • pre-commit hook for your project to run wily
    • CONTRIBUTING.md that includes code of conduct
    • HISTORY.md with a nice format
    • tests using pytest
Michael #2: Latest VS Code has Juypter support
  • In this release, closed a total of 49 issues, including:
    • Jupyter support: import notebooks and run code cells in a Python Interactive window
    • Use new virtual environments without having to restart Visual Studio Code
    • Code completions in the debug console window
    • Improved completions in language server, including recognition of namedtuple, and generic types
  • The extension now contains new editor-centric interactive programming capabilities built on top of Jupyter.
  • have Jupyter installed in your environment (e.g. set your environment to Anaconda) and type #%% into a Python file to define a Cell. You will notice a “Run Cell” code lens will appear above the #%% line:
  • Cells in the Jupyter Notebook will be converted to cells in a Python file by adding #%% lines. You can run the cells to view the notebook output in Visual Studio code, including plots
Brian #3: API Evolution the Right Way
  • A. Jesse Jiryu Davis
  • adding features
  • removing features
  • adding parameters
  • changing behavior
Michael #4: PySimpleGUI now on Qt
  • Project by Mike B
  • Covered back on https://pythonbytes.fm/episodes/show/90/a-django-async-roadmap
  • Simple declarative UI “builder”
  • Looking to take your Python code from the world of command lines and into the convenience of a GUI?
  • Have a Raspberry Pi with a touchscreen that's going to waste because you don't have the time to learn a GUI SDK?
  • Look no further, you've found your GUI package.
  • Now supports Qt
  • Modern Python only
  • More frameworks likely coming
Brian #5: Comparison of the 7 governance PEPs
  • Started by Victor Stinner
  • The different PEPs are compared by:
    • hierarchy
    • number of people involved
    • requirements for candidates to be considered for certain positions
    • elections: who votes, and how
    • term limits
    • no confidence vote
    • teams/experts
    • PEP process
    • core dev promotion and ejection
    • how governance will be updated
    • code of conduct
  • PEP 8000, Python Language Governance Proposal Overview:
    • PEP 8010 - The Technical Leader Governance Model
    • continue status quo (ish)
    • PEP 8011 - Python Governance Model Lead by Trio of Pythonistas
    • like status quo but with 3 co-leaders
    • PEP 8012 - The Community Governance Model
    • no central authority
    • PEP 8013 - The External Governance Model
    • non-core oversight
    • PEP 8014 - The Commons Governance Model
    • core oversight
    • PEP 8015 - Organization of the Python community
    • push most decision-making to teams
    • PEP 8016 - The Steering Council Model
    • bootstrap iterating on governance
Michael #6: Shiboken (from Qt for Python project)
  • From PySide2 (AKA Qt for Python) project
  • Generate Python bindings from arbitrary C/C++ code
  • Has a Typesystem (based on XML) which allows modifying the obtained information to properly represent and manipulate the C++ classes into the Python World.
  • Can remove and add methods to certain classes, and even modify the arguments of each function, which is really necessary when both C++ and Python collide and a decision needs to be made to properly handle the data structures or types.
  • Qt for Python: under the hood
  • Write your own Python bindings
  • Other options include:
Extras:


Audio Download

Posted on 17 November 2018 | 8:00 am


#103 Getting to 10x (results for developers)

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: FEniCS
  • “FEniCS is a popular open-source (LGPLv3) computing platform for solving partial differential equations (PDEs). FEniCS enables users to quickly translate scientific models into efficient finite element code. With the high-level Python and C++ interfaces to FEniCS, it is easy to get started, but FEniCS offers also powerful capabilities for more experienced programmers. FEniCS runs on a multitude of platforms ranging from laptops to high-performance clusters.”
  • Solves partial differential equations efficiently with a combination of C++ and Python.
  • Can be run on a desktop/laptop or deployed to a supercomputer with thousands of parallel processes.
  • is a NumFOCUS fiscally supported project
  • “makes the implementation of the mathematical formulation of a system of partial differential equations almost seamless.” - Sébastien Brisard
  • “FEniCS is in fact a C++ project with a full-featured Python interface. The library itself generates C++ code on-the-fly, that can be called (on-the-fly) from python. It's almost magical... Under the hood, it used to use SWIG, and recently moved to pybind11. I guess the architecture that was set up to achieve this level of automation might be useful in other situations.” - Sébastien Brisard
Michael #2: cursive_re
  • via Christopher Patti, created by Bogdan Popa
  • Readable regular expressions for Python 3.6 and up.
  • It’s a tiny Python library made up of combinators that help you write regular expressions you can read and modify six months down the line.
  • Best understood via an example:
    >>> hash = text('#')
    >>> hexdigit = any_of(in_range('0', '9') + in_range('a', 'f') + in_range('A', 'F'))
    >>> hexcolor = (
    ...     beginning_of_line() + hash +
    ...     group(repeated(hexdigit, exactly=6) | repeated(hexdigit, exactly=3)) +
    ...     end_of_line()
    ... )
    >>> str(hexcolor)
    '^\\#([a-f0-9]{6}|[a-f0-9]{3})$'
  • Has automatic escaping for [ and \ etc: str(any_of(text("[]"))) → '[\\[\\]]'
  • Easily testable / inspectable. Just call str on any expression.
Brian #3: pyimagesearch Michael #4: Visualization of Python development up till 2012
  • via Ophion Group (on twitter)
  • mercurial (hg) source code repository commit history
  • August 1990 - June 2012 (cpython 3.3.0 alpha)
  • Watch the first minute, then click ahead minute at a time and watch for a few seconds to get the full feel
  • Really interesting to see a visual representation of the growth of an open source ecosystem
  • Built with Gource: https://gource.io/
  • Who wants to build this for 2012-present?
  • Would make an amazing lightning talk!
Brian #5: Getting to 10x (Results): What Any Developer Can Learn from the Best
  • Forget the “10x” bit if that term is fighting words. - Brian’s advice
    • How about just “ways to improve your effectiveness as a developer”?
  • “… there is a clear path to excellence. People aren’t born great developers. They get there through focused, deliberate practice.”
  • traits of great developers
    • problem solver
    • skilled
    • mentor/teacher
    • excellent learner
    • passionate
  • traits to avoid:
    • incompetent
    • arrogant
    • uncooperative
    • unmotivated
    • stubborn
  • Focus on your strengths more than your weaknesses
  • Pick 1 thing to improve on this week and focus on it relentlessly
Michael #6: Chaos Toolkit
  • Chaos Engineering is the discipline of experimenting on a distributed system in order to build confidence in the system's capability to withstand turbulent conditions in production.
  • Netflix uses the chaos monkey (et. al.) on their systems. Covered on https://talkpython.fm/episodes/show/16/python-at-netflix
  • The Chaos Toolkit aims to be the simplest and easiest way
  • to explore building, and automating, your own Chaos Engineering Experiments.
  • Integrates with Kubernetes, AWS, Google Cloud, Microsoft Azure, etc.
  • To give you an idea, here are some things it can do to aws:
    • lambda: delete_function_concurrency Removes concurrency limit applied to the specified Lambda
    • stop_instance Stop a single EC2 instance. You may provide an instance id explicitly or, if you only specify the AZ, a random instance will be selected.
Extras:


Audio Download

Posted on 8 November 2018 | 8:00 am


#102 Structure of a Flask Project

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: QuantEcon

  • “Open source code for economic modeling”
  • “QuantEcon is a NumFOCUS fiscally sponsored project dedicated to development and documentation of modern open source computational tools for economics, econometrics, and decision making.”
  • Educational resource that includes:
    • Lectures, workshops, and seminars
    • Cheatsheets for scientific programming in Python and Julia
    • Notebooks
    • QuantEcon.py : open source Python code library for economics

Michael #2: Structure of a Flask Project

  • Flask is very flexible, it has no certain pattern of a project folder structure. Here are some suggestions.
  • I always keep this one certain rule when writing modules and packages:
    • “Don't backward import from root __init__.py.”
  • Candidate structure:
    project/
      __init__.py
      models/
        __init__.py
        users.py
        posts.py
        ...
      routes/
        __init__.py
        home.py
        account.py
        dashboard.py
        ...
      templates/
        base.html
        post.html
        ...
      services/
        __init__.py
        google.py
        mail.py
  • Love it! To this, I would rename routes to views or controllers and add a viewmodels folder and viewmodels themselves.
  • Brian, see anything missing?
    • ya. tests. :)
  • Another famous folder structure is app based structure, which means things are grouped bp application
  • I (Michael) STRONGLY recommend Flask blueprints

Brian #3: Overusing lambda expressions in Python

  • lambda expressions vs defined functions
    1. They can be immediately passed around (no variable needed)
    2. They can only have a single line of code within them
    3. They return automatically
    4. They can’t have a docstring and they don’t have a name
    5. They use a different and unfamiliar syntax
  • misuses:
    • naming them. Just write a function instead
    • calling a single function with a single argument : just use that func instead
  • overuse:
    • if they get complex, even a little bit, they are hard to read
    • has to be all on one line, which reduces readibility
    • map and filter : use comprehensions instead
    • using custom lambdas instead of using operators from the operator module.

Michael #4: Asyncio in Python 3.7

  • by Cris Medina
  • The release of Python 3.7 introduced a number of changes into the async world.
    • Some may even affect you even if you don’t use asyncio.
  • New Reserved Keywords: The async and await keywords are now reserved.
    • There’s already quite a few modules broken because of this. However, the fix is easy: rename any variables and parameters.
  • Context Variables: Version 3.7 now allows the use of context variables within async tasks. If this is a new concept to you, it might be easier to picture it as global variables whose values are local to the currently running coroutines.
  • Python has similar constructs for doing this very thing across threads. However, those were not sufficient in async-world
  • New asyncio.run() function
    • With a call to asyncio.run(), we can now automatically create a loop, run a task on it, and close it when complete.
  • Simpler Task Management: Along the same lines, there’s a new asyncio.create_task() function that helps make tasks that inside the current loop, instead of having to get the loop first and calling create task on top of it.
  • Simpler Event Loop Management: The addition of asyncio.get_running_loop() will help determine the active event loop, and catch a RuntimeError if there’s no loop running.
  • Async Context Managers: Another quality-of-life improvement. We now have the asynccontextmanager() decorator for producing async context managers without the need for a class that implements __aenter__() or __aexit__().
  • Performance Improvements: Several functions are now optimized for speed, some were even reimplemented in C. Here’s the list:
    • asyncio.get_event_loop() is now 15 times faster.
    • asyncio.gather() is 15% faster.
    • asyncio.sleep() is two times faster when the delay is zero or negative.
    • asyncio.Future callback management is optimized.
    • Reduced overhead for asyncio debug mode.
  • Lots lots more

Brian #5: Giving thanks with **pip thank**

Michael #6: Getting Started With Testing in Python

  • by Anthony Shaw, 33 minutes reading time according to Instapaper
  • Automated vs. Manual Testing
  • Unit Tests vs. Integration Tests: A unit test is a smaller test, one that checks that a single component operates in the right way. A unit test helps you to isolate what is broken in your application and fix it faster.
  • Compares unittest, nose or nose2, pytest
  • Covers things like:
    • Writing Your First Test
    • Where to Write the Test
    • How to Structure a Simple Test
    • How to Write Assertions
    • Dangers of Side Effects
  • Testing in PyCharm and VS Code
  • Testing for Web Frameworks Like Django and Flask
  • Advanced Testing Scenarios
  • Even: Testing for Security Flaws in Your Application

Extras:


Audio Download

Posted on 31 October 2018 | 8:00 am


#101 Nobel Prize awarded to a Python convert

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: Asterisks in Python: what they are and how to use them

  • I just ** love *s
  • Using * and ** to pass arguments to a function
    • * for list, ** for keyword arguments from a dictionary
  • Using * and ** to capture arguments passed into a function
  • Using * to accept keyword-only arguments
  • Using * to capture items during tuple unpacking
    • you can capture the rest if you only want to grab a few
  • Using * to unpack iterables into a list/tuple
  • Using ** to unpack dictionaries into other dictionaries

Michael #2: responder web framework

  • From Kenneth Reitz — A familiar HTTP Service Framework
  • Already has 1,393 github stars
  • Flask-like but with async support and
    • A pleasant API, with a single import statement.
    • Class-based views without inheritance.
    • ASGI framework, the future of Python web services.
    • WebSocket support!
    • The ability to mount any ASGI / WSGI app at a subroute.
    • f-string syntax route declaration.
    • Mutable response object, passed into each view. No need to return anything.
    • Background tasks, spawned off in a ThreadPoolExecutor.
    • GraphQL (with GraphiQL) support!
    • OpenAPI schema generation.
    • Single-page webapp support
  • Responder gives you the ability to mount another ASGI / WSGI app at a subroute
  • uvicorn: powers responder and is built on top of uvloop
  • asgi: https://www.encode.io/articles/hello-asgi/

Brian #3: Python Example resource: pythonprogramming.in

  • Lots of examples
  • Python basics including date time, strings, dictionaries
  • pandas, matplotlib, tensorflow basics
  • data structures and algorithms
  • Nice reference, especially for people getting into Python for data science or scientific work.

Michael #4: This year’s Nobel Prize in economics was awarded to a Python convert**

  • Nordhaus and Romer “have designed methods that address some of our time’s most fundamental and pressing issues: long-term sustainable growth in the global economy and the welfare of the world’s population,”
  • Notably for a 62-year-old economist of his distinction, he is a user of the programming language Python.
  • Romer believes in making research transparent. He argues that openness and clarity about methodology is important for scientific research to gain trust.
  • He tried to use Mathematica to share one of his studies in a way that anyone could explore every detail of his data and methods. It didn’t work. He says that Mathematica’s owner, Wolfram Research, made it too difficult to share his work in a way that didn’t require other people to use the proprietary software, too.
  • Romer believes that open-source notebooks are the way forward for sharing research. He believes they support integrity, while proprietary software encourage secrecy. “The more I learn about proprietary software, the more I worry that objective truth might perish from the earth,” he wrote.
  • Michael covered a similar story for the Nobel Prize in Physics at CERN on Talk Python
  • Jake Vanderplas Keynote at PyCon 2017: “The unexpected effectiveness of Python in Science”

Brian #5: More in depth TensorFlow

Michael #6: MAKERphone - an educational DIY mobile phone

  • MAKERphone is an educational DIY mobile phone designed to bring electronics and programming to the crowd in a fun and interesting way.
    • A fully functional mobile phone that you can code yourself
    • Games such as space invaders, pong, or snake
    • Apps such as a custom media player that only plays cat videos
    • Programs in Arduino
    • Lines of code in Python
    • Your first working piece of code in Scratch
    • A custom case

Extras:


Audio Download

Posted on 24 October 2018 | 8:00 am