Python Bytes

by Michael Kennedy and Brian Okken

Python Bytes is a weekly podcast hosted by Michael Kennedy and Brian Okken. The show is a short discussion on the headlines and noteworthy news in the Python, developer, and data science space.

  

Latest Episodes

#138 Will PyOxidizer weld shut one of Python's major gaps?

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: flake8-comprehensions

  • submitted by Florian Dahlitz
  • I’m already using flake8, so adding this plugin is a nice idea.
  • checks your code for some generator and comprehension questionable code.
    • C400 Unnecessary generator - rewrite as a list comprehension.
    • C401 Unnecessary generator - rewrite as a set comprehension.
    • C402 Unnecessary generator - rewrite as a dict comprehension.
    • C403 Unnecessary list comprehension - rewrite as a set comprehension.
    • C404 Unnecessary list comprehension - rewrite as a dict comprehension.
    • C405 Unnecessary (list/tuple) literal - rewrite as a set literal.
    • C406 Unnecessary (list/tuple) literal - rewrite as a dict literal.
    • C407 Unnecessary list comprehension - '[HTML_REMOVED]' can take a generator.
    • C408 Unnecessary (dict/list/tuple) call - rewrite as a literal.
    • C409 Unnecessary (list/tuple) passed to tuple() - (remove the outer call to tuple()/rewrite as a tuple literal).
    • C410 Unnecessary (list/tuple) passed to list() - (remove the outer call to list()/rewrite as a list literal).
    • C411 Unnecessary list call - remove the outer call to list().
  • Example:
    • Rewrite list(f(x) for x in foo) as [f(x) for x in foo]
    • Rewrite set(f(x) for x in foo) as {f(x) for x in foo}
    • Rewrite dict((x, f(x)) for x in foo) as {x: f(x) for x in foo}

Michael #2: PyOxidizer (again)

  • Michael’s assessment - There are three large and looming threats to Python. Lack of
    • A real mobile development story
    • GUI applications on desktop operating systems
    • Sharing your application with users (this is VERY far from deployment to servers)
  • Cover PyOxidizer before but seems to have just rocketed off last couple of weeks.
  • At their PyCon 2019 keynote talk, Russel Keith-Magee identified code distribution as a potential black swan - an existential threat for longevity - for Python.
    • Python hasn't ever had a consistent story for how I give my code to someone else, especially if that someone else isn't a developer and just wants to use my application.
  • They announced the first release of PyOxidizer (project, documentation), an open source utility that aims to solve the Python application distribution problem!
  • PyOxidizer's marquee feature is that it can produce a single file executable containing a fully-featured Python interpreter, its extensions, standard library, and your application's modules and resources.
  • You can have a single .exe providing your application.
  • Unlike other tools in this space which tend to be operating system specific, PyOxidizer works across platforms (currently Windows, macOS, and Linux - the most popular platforms for Python today).
  • PyOxidizer loads everything from memory and there is no explicit I/O being performed. When you **import** a Python module, the bytecode for that module is being loaded from a memory address in the executable using zero-copy.
  • This makes PyOxidizer executables faster to start and import - faster than a python executable itself!

Brian #3: Using changedir to avoid the need for src

  • I’ve been experimenting with combining flit, pytest, tox, and coverage for new projects.
  • And in doing so, ran across a cool feature of tox that I didn’t know about before, changedir.
  • It’s a feature of tox to allow you to run tests in a different directory than the top level project directory.
  • I talk about this more in episode 80 of Test & Code.
    • As an example project I build yet another markdown converter using regular expressions.
    • This is funny to me, considering the recent cloudflare outage due to a single regular expression. https://blog.cloudflare.com/cloudflare-outage/
    • “Tragedy is what happens to me, comedy is what happens to you” - Mel Brooks approximate quote.

Michael #4: WebRTC and ORTC implementation for Python using asyncio

  • Web Real-Time Communication (WebRTC) - WebRTC is a free, open project that provides browsers and mobile applications with Real-Time Communications (RTC) capabilities via simple APIs.
  • Object Real-Time Communication (ORTC) - ORTC (Object Real-Time Communications) is an API allowing developers to build next generation real-time communication applications for web, mobile, or server environments.
  • The API closely follows its Javascript counterpart while using pythonic constructs:
    • promises are replaced by coroutines
    • events are emitted using pyee.EventEmitter
  • The main WebRTC and ORTC implementations are either built into web browsers, or come in the form of native code.
  • In contrast, the aiortc implementation is fairly simple and readable.
    • Good starting point for programmers wishing to understand how WebRTC works or tinker with its internals.
    • Easy to create innovative products by leveraging the extensive modules available in the Python ecosystem.
    • For instance you can build a full server handling both signaling and data channels or apply computer vision algorithms to video frames using OpenCV.

Brian #5: Apprise - Push Notifications that work with just about every platform!

  • listener suggestion
  • cool shim project to allow multiple notification services in one app
  • Apprise allows you to send a notification to almost all of the most popular notification services available to us today such as: Telegram, Pushbullet, Slack, Twitter, etc.
    • One notification library to rule them all.
    • A common and intuitive notification syntax.
    • Supports the handling of images (to the notification services that will accept them).”
  • supports
    • notification services such as discord, gitter, ifttt, mailgun, mattermost, MS teams, twitter, …
    • SMS notification through Twilio, Nexmo, AWS, D7
    • email notifications

Michael #6: Websauna web framework

  • Websauna is a full stack Python web framework for building web services and back offices with admin interface and sign up process https://websauna.org
  • "We have web applications 80% figured out. Websauna takes it up to 95%.
  • Built upon Python 3, Pyramid, and SQLAlchemy.
  • When to use it?
    • Websauna is focused on Internet facing sites where you have a public or private sign up process and an administrative interface. Its sweet spots include custom business portals and software-as-a-service products which are too specialized for off-the-shelf solutions.
  • Benefits
    • Focus on core business logic as Websauna provides basic website building blocks like sign up and sign in.
    • Low learning curve and friendly comprehensive documentation help novice developers
    • Emphasis is on meeting business requirements with reliable delivery times, responsiveness, consistency
    • Site operations is half the story. Websauna provides an automated deployment process and integrates with monitoring, security and other DevOps solutions.

Extras

Michael:

Brian:

  • Recent Test & Code episodes were solo because I’m in the middle of a work move and didn’t want to schedule interviews around a crazy work schedule. However, that should settle down in July and I can get back to getting great guests on the show. But I’m also having fun with solo topics, so I’ll keep that in the mix.
    • upshot: if I’ve contacted you or you me about being on the show and you haven’t heard from me lately, give me a nudge with a DM or email or something.

Jokes

  • An SQL query goes into a bar, walks up to two tables and asks, 'Can I join you?'
  • Not a joke, really, but along the lines of “comedy when it happens to you”.


Audio Download

Posted on 8 July 2019 | 8:00 am


#137 Advanced Python testing and big-time diffs

Sponsored by Rollbar: https://pythonbytes.fm/rollbar

Brian #1: Comparing the Same Project in Rust, Haskell, C++, Python, Scala and OCaml

  • Tristan Hume, writing about a university project
  • Teams of up to 3 people, multi month, write a Java to x86 compiler in language of choice
  • Needed to pass both known and unknown tests.
  • Secret tests to be run after submission encouraged teams to add more testing than provided.
  • Nothing but standard libraries, and no parsing libraries, even if in standard.
  • Lines of code
    • Rust baseline
    • Haskell: 1-1.6x
    • C++: 1.4x
    • Rust (another team): 3x
    • Scala: 0.7 x
    • OCaml: 1-1.6x
    • Python: about half the size
  • Python version
    • one person
    • used metaprogramming
    • more extra features than any other team
    • passed all public and secret tests

Michael #2 : Pylustrator is a program to style your matplotlib plots

  • via Len Wanger
  • Pylustrator is a program to style your matplotlib plots for publication.
  • Subplots can be resized and dragged around by the mouse, text and annotations can be added.
  • Changes can be saved to the initial plot file as python code.

Brian #3: MongoDB 4.2

  • Distributed Transactions
    • extends multi-document ACID transactions across documents, collections, dbs in a replica set, and sharded cluster.
  • Field Level Encryption
    • encryption done on client side
    • satisfies GDPR by allowing customer key destruction rendering server data on customer useless.
    • system administration can be done with no exposure to private data

Michael #4: Deep Difference and search of any Python object/data

  • via François Leblanc
  • DeepDiff: Deep Difference of dictionaries, iterables, strings and other objects. It will recursively look for all the changes.
  • Lots of nice touches:
    • List difference ignoring order or duplicates
    • Report repetitions
    • Exclude certain types from comparison
    • Exclude part of your object tree from comparison
    • Significant Digits
  • DeepSearch: Search for objects within other objects.
  • DeepHash: Hash of ANY python object based on its contents even if the object is not considered hashable! DeepHash is supposed to be deterministic in order to make sure 2 objects that contain the same data, produce the same hash.

Brian #5: Advanced Python Testing

  • Josh Peak
  • “This article is mostly for me to process my thoughts but also to pave a path for anyone that wants to follow a similar journey on some more advanced python testing topics.”
  • Learning journey (including some great podcasts and an awesome book on testing)
  • Testing tools
    • basic test structure
    • adding black to testing with pytest-black
    • linting with pylint
      • including a very cool speed up trick to only lint modified files.
    • flake8, including docstring checking
    • tox.ini modifications
    • code coverage goals and how to ratchet up to that goal with --cov-fail-under
      • cool learning: “Increase code coverage by testing more code OR deleting code.”
    • fixtures for database connections
    • utilizing mocks, spies, stubs, and monkey patches, including pytest-mock
    • pytest-vcr to save network interactions and replay them in future test runs, resulting in a 10x speedup.
  • Lots of links and tangents possible from this article.

Michael #6: Understanding Python's del

  • via Kevin Buchs
  • Official docs
  • General confusion of what this does
  • Looks like memory management, and it mostly isn’t
  • Primary use: remove an item from a list given its index instead of its value or from a dictionary given its key: del person['profession'] # person is a dict
  • del statement can also be used to remove slices from a list del lst[2:4]
  • del can also be used to delete entire variables: del variable
  • Recently covered how The CPython Bytecode Compiler is Dumb. Proactive dels could help.

Extras

Michael:

Jokes

Optimist: The glass is half full. Pessimist: The glass is half empty. Programmer: The glass is twice as large as necessary.

Pragmatist: allowing room for requirements oversights, scope creep, and schedule overrun.

From “The Upside” with Kevin Hart and Bryan Cranston (watched it last night): K: Would you invest in [HTML_REMOVED]? B: That seems too niche. K: What’s “niche” mean? B: It’s the girl version of “nephew”.


Audio Download

Posted on 2 July 2019 | 8:00 am


#136 A Python kernel rather than cleaning the batteries?

Brought to you by Datadog: pythonbytes.fm/datadog

Brian #1: Voilà!

  • “from Jupyter notebooks to standalone applications and dashboards”
  • Turn a notebook into a web app with:
    • custom widgets
    • runnable code (but not editable)
    • interactive plots
    • different custom grid layouts
    • templates

Michael #2: Toward a “Kernel Python”

  • By Glyph
  • Glyph wants to Marie Kondō the standard library (and I think I agree with him)
  • We have PEP 594 for removing obviously obsolete and unmaintained detritus from the standard library.
  • PEP 594 is great news for Python, and in particular for the maintainers of its standard library, who can now address a reduced surface area.
  • Believes the PEP may be approaching the problem from the wrong direction.
  • One “dead” battery is the colorsys module: why not remove it? “The module is useful to convert CSS colors between coordinate systems. Today, however, the modules you need to convert colors between coordinate systems are only a pip install away.
  • Every little bit is overhead for the core devs, consider the state of PRs
  • Looking at CPython’s keyword-based review queue, we can see that there are 429 tickets currently awaiting review. The oldest PR awaiting review hasn’t been touched since February 2, 2018, which is almost 500 days old.
  • By Glyph’s subjective assessment, on this page of 25 PRs, 14 were about the standard library, 10 were about the core language or interpreter code
  • We need a “kernel” version of Python that contains only the most absolutely minimal library, so that all implementations can agree on a core baseline that gives you a “python”
  • Michael: There will be a cost to beginners. But there is already.

Brian #3: Use __main__.py

  • I didn’t know it was that easy to get python -m [HTML_REMOVED] to work.

Michael #4: The CPython Bytecode Compiler is Dumb

  • by Chris Wellons
  • Given multiple ways to express the same algorithm or idea, Chris tends to prefer the one that compiles to the more efficient bytecode.
  • Fortunately CPython, the main and most widely used implementation of Python, is very transparent about its bytecode. It’s easy to inspect and reason about its bytecode. The disassembly listing is easy to read and understand.
  • One fact has become quite apparent: the CPython bytecode compiler is pretty dumb. With a few exceptions, it’s a very literal translation of a Python program, and there is almost no optimization.
  • Darius Bacon points out that Guido van Rossum himself said, “Python is about having the simplest, dumbest compiler imaginable.” So this is all very much by design.
  • The consensus seems to be that if you want or need better performance, use something other than Python. (And if you can’t do that, at least use PyPy.) ← Cython people, Cython.
  • Example
    def foo():
        x = 0
        y = 1
        return x

Could easily be:

    def foo():
        return 0

Yet, CPython completely misses this optimization for both x and y:

      2           0 LOAD_CONST               1 (0)
                  2 STORE_FAST               0 (x)
      3           4 LOAD_CONST               2 (1)
                  6 STORE_FAST               1 (y)
      4           8 LOAD_FAST                0 (x)
                 10 RETURN_VALUE

And so on.

  • Brett Cannot has expressed performance as a major focus for CPython, maybe there is something here?

Brian #5: You can play with EdgeDB now, maybe

Michael #6: 16 Python libraries that helped a healthcare startup grow

  • via Waqas Younas
  • Worked with a U.S.-based healthcare startup for 7 years. This startup developed a software product that sent appointment reminders to the patients of healthcare facilities; the reminders were sent via email, text, and IVR.
  1. Paramiko - A Python implementation of SSHv2.
  2. built-in CSV module
  3. SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper
  4. Requests - HTTP for Humans™
  5. BeautifulSoup - Python library for pulling data out of HTML and XML files.
  6. testscenarios - a pyunit extension for dependency injection
  7. HL7 - a simple library for parsing messages of Health Level 7 (HL7) version 2.x into Python objects.
  8. Python-Phonenumbers - Library for parsing, formatting, and validating international phone numbers
  9. gevent - a coroutine -based Python networking library that uses greenlet to provide a high-level synchronous API on top of the libev or libuv event loop.
  10. dateutil - powerful extensions to datetime (pip install python-dateutil)
  11. Matplotlib - a Python 2D plotting library which produces publication quality figures
  12. python-magic - a python interface to the libmagic file type identification library. libmagic identifies file types by checking their headers according to a predefined list of file types.
  13. Django - a high-level Python Web framework that encourages rapid development and clean, pragmatic design
  14. Boto - a Python package that provides interfaces to Amazon Web Services.
  15. Mailgun Python bindings - helped us send appointment reminders seamlessly
  16. Twilio’s Python bindings - helped us send appointment reminders seamlessly

Extras

Michael:

United States Digital Service

Jokes

Difference between ML & AI? Ans.


Audio Download

Posted on 25 June 2019 | 8:00 am


#135 macOS deprecates Python 2, will stop shipping it (eventually)

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Special guest Max Sklar

Brian #1: Why do Python lists let you += a tuple, when you can’t + a tuple?

  • Reuven Lerner
    >>> x = [1, 2, 3]
    >>> b = (4, 5, 6)
    >>> x + b
    Traceback (most recent call last):
      File "[HTML_REMOVED]", line 1, in [HTML_REMOVED]
    TypeError: can only concatenate list (not "tuple") to list
    >>> x += b
    >>> x
    [1, 2, 3, 4, 5, 6]
  • Huh??
  • “It turns out that the implementation of list.__iadd__ (in place add) takes the second (right-hand side) argument and adds it, one element at a time, to the list. It does this internally, so that you don’t need to execute any assignment after. The second argument to “+=” must be iterable.”

Max #2: R vs Python, R is out of top 20 languages despite statistical boom

  • Subtitle: is R declining because of Python?
  • First of all, this article is about an index on the popularity of programming languages from an organization TIOBE. They have an index on the popularity of programming languages. Obviously it’s a combination of many different scores, and that could be controversial, but I’m going to assume that they put some thought into how the rankings are calculated, and that it’s as good as any.
  • A few stories here: first Python hit at all time high in their ranking at number 3, beating out c++ I believe for the first time, and only Java and C are above it.
  • The other story is that the statistical language R dipped below 20 to number 21, and the speculation is that Python has sort of taken over as the preferred statistical language to R.
  • Personally, I got into Python much sooner, because I started as a software engineer, and moved into data science and machine learning. So after taking CS, and programming in Java and C for a few years, python came much more naturally.
  • But still - a lot of people who are data-science first (and they have an additional skills to the kind of hybrid that I am) like and prefer R, and they can use it in a specialized way and get good results.
  • Personally, I’m going to stick with python, because there’s so many statistical libraries yet to learn, and it’s served me well thus far.
  • The language I’ve used most in recent years, Scala, is surprisingly down at 31 - not even close!
  • related: https://www.zdnet.com/article/programming-languages-python-predicted-to-overtake-c-and-java-in-next-4-years/

Michael #3: macOS deprecates Python 2, will stop shipping it (eventually)

  • via Dan Bader, on the heels of WWDC 2019
  • “Future versions of macOS won’t include scripting language runtimes by default”
  • Contrast this with Windows just now starting to ship with Python 3
  • In the same announcement:
  • “Use of Python 2.7 isn’t recommended as this version is included in macOS for compatibility with legacy software. Future versions of macOS won’t include Python 2.7. Instead, it’s recommended that you run python3 from within Terminal. (51097165)”
  • Also has impact wider than “us”. E.g. No Ruby or Perl, means home brew doesn’t install easily which is how we get Python 3!

Brian #4: Pythonic Ways to Use Dictionaries

  • Al Sweigart
  • A few pythonic uses of dictionaries that are not obvious to new people.
  • Use get() and setdefault() with Dictionaries
    • get(key, default=[HTML_REMOVED]) allows you to read a key without checking for it’s existence beforehand.
    • setdefault(key, default=[HTML_REMOVED]) is a bit of a strange duck but still useful. Set the value of something if it doesn’t exist yet.
  • Python Uses Dictionaries Instead of a Switch Statement
    • Just do it a few times to get the hang of it. Then it becomes natural.
  • Michael's switch addition for Python: https://github.com/mikeckennedy/python-switch

Max #5: Things you are probably not using in Python 3 But Should

  • This is from Datawhatnow.com
  • This is particularly relevant for me, since I used python legacy at Foursquare for many years, and now coming back to it taking another look at python v3.
  • One that looks very useful is f-Strings where you can put the variable name in braces in a string and just have it replaced. I’ve seen things like this in other languages - notably PHP and most front-end scripts. Makes the code very readable.
    • Except I know I’m going to screw up by leaving out that stray “f” in front of the string. It should almost be automatic, because how often are you putting these variable names in braces?
  • Another thing I didn’t know python 3 had - again I’m kind of just get started with python 3 is enumerations.
  • I’ve been using Enums for years in scala (really case classes) to make my code WAY more readable. Will keep that in mind when developing in python 3.

Michael #6: Have a time machine? C++ would get the Python 2 → 3 treatment too

  • via James Small
  • In a recent CppCast interview, Herb Sutter describes how he would change C/C++ types if he could go back in time.
  • This is almost exactly how things were changed from Python 2 to Python 3 (str split into Unicode strings and byte arrays)
  • So my question to you two is: Why was the transition so hard? Was it just habit and stubbornness? What could the PSF have done?

Extras

Michael:

  • pip install mystery
    • by Divo Kaplan
    • A random Python package every time.
    • Mystery is a Python package that is instantiated as a different package every time you install it!
    • Inspired by one of our episodes
  • Get our effective pycharm book bundle with the courses over at effectivepycharm.com

Brian:

Max:

  • The Local Maximum
    • Weekly Podcast that covers both the theoretical issues in probability theory, philosophy, and machine learning, but then applies it in a practical way to things like current events and product development.
    • For example, a few weeks ago I did a show on how to estimate the probably of an event that has never occurred
    • We also cover things like Apple’s decision to breakup iTunes, how the internet is shaping up in places like Cuba, and the controversy around YouTube’s recommendation algorithm.

Jokes

MK: There are only two hard problems in Computer Science: cache invalidation, naming things and off-by-one-errors.


Audio Download

Posted on 20 June 2019 | 8:00 am


#134 Python proves Mercury is the closest planet to Earth

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: Three scientists publish a paper proving that Mercury, not Venus, is the closest planet to Earth. using Python

  • contributed by, and explained by, listener Andrew Diederich.

    “This is from the March 19th, 2019 Strange Maps article. Which planet is, on average, closest to the Earth? Answer: Mercury. Actually, Mercury is, on average, the closest to all other planets, because it’s closest to the sun.”

  • article, including video, uses PyEphem, which apparently is now deprecated and largely replaced with skyfield.

Michael #2: Github semantics

  • Parsing, analyzing, and comparing source code across many languages
  • Written in a Haskell, it’s a library and command line tool for parsing, analyzing, and comparing source code.
  • It’s still early days yet, but semantic can do a lot of cool things, and is powering public-facing GitHub features. I’m tremendously excited as to see how it’ll evolve now that it’s a community-facing project.
  • Understands: Python, TypeScript, JavaScript, Ruby, Go, …
  • here are some cool things inside it:
    • A flow-sensitive, caching, generalized interpreter for imperative languages
    • An abstract interpreter that generates scope graphs for a given program text
    • A strategic rewriting system based on recursion schemes for open syntax terms

Brian #3: flake8-black

  • Contributed by Nathan Clayton
  • “The point of this plugin is to be able to run black --check ... from within the flake8 plugin ecosystem.”
  • I like to run flake8 during development both to keep things neat, and to train myself to just write code in a more standard way. This is a way to run black with no surprises.

Michael #4: Python Preview for VS Code

  • You write Python code (script style mostly), it creates an object-visualization
  • Think of a picture your first year C++ CS prof might draw. This extension does that automatically as you write Python code
  • Looks to be based (conceptually) on Philip Guo’s Python Tutor site.

Brian #5: Create and Publish a Python Package with Poetry

  • John Franey
  • Walks through creating a package, customizing the pyproject.toml, and talks about the different settings in the toml and what it means.
  • Then using the testpypi, and finally publish.

Michael #6: Pointers in Python: What's the Point?

  • by Logan Jones
  • Quick question: Does Python have pointers (outside of C-extensions, etc of course)?
  • Yet Python is more pointer heavy than most languages (more so than C# more so than even C++)!
  • In Python, everything is an object, even numbers and booleans.
  • Each object contains at least three pieces of data:
    • Reference count
    • Type
    • Value
  • Check that you have the same object is instead of ==
  • Python variables are pointers, just safe ones.
  • Interesting little tidbit from the article:
    • Interning strings is useful to gain a little performance on dictionary lookup—if the keys in a dictionary are interned, and the lookup key is interned, the key comparisons (after hashing) can be done by a pointer compare instead of a string compare. (Source)
  • But like we have inline-assembly in C++ and unsafe mode in C#, we can use pointers in Cython or more fine-grained with ctypes.

Extras

Michael:

  • PSF needs your help. Spread the word about the fundraiser and please, ask your company to contribute: Building the PSF: the Q2 2019 Fundraiser (Donations are tax-deductible for individuals and organizations that pay taxes in the United States)
    • “Contributions help fund workshops, conferences, pay meetup fees, support fiscal sponsorships, PyCon financial aid, and development sprints. ”

Jokes

via Jay Miller

What did the developer name his newborn boy? JSON


Audio Download

Posted on 12 June 2019 | 8:00 am


#133 Github sponsors - The model open source has been waiting for?

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: Python built-ins worth learning

  • Trey Hunner
  • “I estimate most Python developers will only ever need about 30 built-in functions, but which 30 depends on what you’re actually doing with Python.”
  • “I recommend triaging your knowledge:
    • Things I should memorize such that I know them well
    • Things I should know about so I can look them up more effectively later
    • Things I shouldn’t bother with at all until/unless I need them one day”
  • all 69 built-in functions, split into
    • commonly known
    • overlooked by beginners
    • learn it later
    • maybe learn it eventually
    • you likely don’t need these
  • Highlighting some:
    • overlooked by beginners
      • sum, enumerate, zip, bool, reversed, sorted, min, max, any, all
    • know it’s there, but learn it later:
      • open, input, repr, super, property, issubclass, isinstance, hasattr, getattr, setattr, delattr, classmethod, staticmethod, next
  • my notes
    • I think getattr should be learned early on, because it’s default behavior is so useful. But can’t use it for dicts. Use mydict.get(key, default) for dictionaries.

Michael #2: Github sponsors and match

  • Like Patreon but for GitHub projects
  • 2x your sponsorship: Github matches! To boost community funding, we'll match contributions up to $5,000 during a developer’s first year in GitHub Sponsors with the GitHub Sponsors Matching Fund.
  • 100% to developers, Zero fees: GitHub will not charge fees for GitHub Sponsors.
  • Anyone who contributes to open source—whether through code, documentation, leadership, mentorship, design, or beyond—is eligible for sponsorship.

Brian #3: Build a REST API in 30 minutes with Django REST Framework

  • Bennett Garner
  • Very fast intro including:
    • Set up Django
    • Create a model in the database that the Django ORM will manage
    • Set up the Django REST Framework
    • Serialize the model from step 2
    • Create the URI endpoints to view the serialized data
  • Example is a simple hero db with hero name and alias.

Michael #4: Dependabot has been acquired by GitHub

  • Automated dependency updates: Dependabot creates pull requests to keep your dependencies secure and up-to-date.
  • I personally use and recommend PyUP: https://pyup.io/
  • How it works:
    • Dependabot checks for updates: Dependabot pulls down your dependency files and looks for any outdated or insecure requirements.
    • Dependabot opens pull requests: If any of your dependencies are out-of-date, Dependabot opens individual pull requests to update each one.
    • You review and merge: You check that your tests pass, scan the included changelog and release notes, then hit merge with confidence.
  • Here's what you need to know:
    • We're integrating Dependabot directly into GitHub, starting with security fix PRs 👮‍♂️
    • You can still install Dependabot from the GitHub Marketplace whilst we integrate it into GitHub, but it's now free of charge 🎁
    • We've doubled the size of Dependabot's team; expect lots of great improvements over the coming months 👩‍💻👨‍💻👩‍💻👨‍💻👩‍💻👨‍💻
  • Paid accounts are now free, automatically.

Brian #5: spoof “New features planned for Python 4.0

  • Charles Leifer - also known for Peewee ORM
  • This is funny, but painful. Is it too soon to joke about the pain of 2 to 3?
  • A few of my favorites
    • PEP8 will be updated. Line lengths will be increased to 89.5 characters. (compromise between 79 and 100)
    • All new libraries and standard lib modules must include the phrase "for humans" somewhere in their title.
    • Type-hinting has been extended to provide even fewer tangible benefits and will be called type whispering.
    • You can make stuff go faster by adding async before every other keyword.
    • Notable items left out of 4.0
      • Still no switch statement.
      • No improvements to packaging.

Michael #6: BlackSheep web framework

  • Fast HTTP Server/Client microframework for Python asyncio, using Cython, uvloop, and httptools.
  • Very Flask-like API. Interesting to consider the “popularity” of Flask vs Django in this context.
  • Objectives
  • Also has an async client much like aiohttp.

Extras

Michael:

Jokes

  • How do you generate a random string? Put a first year Computer Science student in Vim and ask them to save and exit.
  • Waiter: He's choking! Is anyone a doctor? Programmer: I'm a Vim user.


Audio Download

Posted on 5 June 2019 | 8:00 am


#132 Algorithms as objects

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: History of CircuitPython

  • PSF blog, A. Jesse Jiryu Davis
  • Adafruit hired Scott Shawcroft to port MicroPython to their SAMD21 chip they use on many of their boards.
  • CircuitPython is a friendly fork of MicroPython. Same licensing, and they share improvements back and forth.
  • “MicroPython customizes its hardware APIs for each chip family to provide speed and flexibility for hardware experts. Adafruit’s audience, however, is first-time coders. Shawcroft said, “Our goal is to focus on the first five minutes someone has ever coded.” “
  • “Shawcroft aims to remove all roadblocks for beginners to be productive with CircuitPython. As he demonstrated, CircuitPython auto-reloads and runs code when the user saves it; there are two more user experience improvements in the latest release. First, serial output is shown on a connected display, so a program like print("hello world") will have visible output even before the coder learns how to control LEDs or other observable effects.”
  • Related: CircuitPython 4.0.0 released

Michael #2: R Risks Python Swallowing It Whole: TIOBE

  • Is the R programming language in serious trouble? According to the latest update of the TIOBE Index, the answer seems to be “yes.”
  • R has finally tumbled out of the top 20 languages
  • “It seems that there is a consolidation going on in the statistical programming market. Python has become the big winner.”
  • Briefly speculates why is Python (which ranked fourth on this month’s list) winning big in data science? My thought: Python is a full spectrum language with solid numerical support.

Brian#3: The Missing Introduction To Containerization

  • Aymen El Amri
  • Understanding containerization through history
    • chroot jail, 1979, allowed isolation of a root process and it’s children from the rest of the OS, but with no security restrictions.
    • FreeBSD Jail, 2000, more secure, also isolating the file system.
    • Linux VServer, 2001, added “security contextes” and used new OS system-level virtualization. Allows you to run multiple Linux distros on a single VPS.
    • Oracle Solaris Containers, 2004, system resource controls and boundary separation provided by “zone”.
    • OpenVZ, 2005, OS-level virtualization. Used by many hosting companies to isolate and sell VPSs.
    • Google’s CGroups, 2007, a mechanizm to limit and isolate resource usage. Was mainlained into Linux kernel the same year.
    • LXC, Linux Containers, 2008, Similar to OpenVX, but uses CGroups.
    • CloudFoundry’s Warden, 2013, an API to manage environments.
    • Docker, 2013, os-level virtualization
    • Google’s LMCTFY (Let me contain that for you), 2014, an OSS version of Google’s container stack, providing Linux application containers. Most of this tech is being incorporated into libcontainer.
    • “Everything at Google runs on containers. There are more than 2 billion containers running on Google infrastructure every week.”
    • CoreOS’s rkt, 2014, an alternative to Docker.
  • Lots of terms defined
    • VPS, Virtual Machine, System VM, Process VM, …
  • OS Containers vs App Containers
  • Docker is both a Container and a Platform

  • This is halfway through the article, and where I got lost in an example on creating a container sort of from scratch. I think I’ll skip to a Docker tutorial now, but really appreciate the back story and mental model of containers.

Michael #4: Algorithms as objects

  • We usually think of an algorithm as a single function with inputs and outputs.
  • Our algorithms textbooks reinforce this notion.
  • They present very concise descriptions that neatly fit in half of a page.
  • Little details add up until you’re left with a gigantic, monolithic function
  • monolithic function lacks readability
  • the function also lacks maintainability
  • Nobody wants to touch this code because it’s such a pain to get any context
  • Complex code requires abstractions
  • How to tell if your algorithm is an object
  • Code smell #1. It’s too long or too deeply nested
  • Code smell #2. Banner comments
  • Code smell #3. Helper functions as nested closures, but it’s still too long
  • Code smell #4. There are actual helper functions, but they shouldn’t be called by anyone else
  • Code smell #5. You’re passing state between your helper functions
  • Write your algorithm as an object
  • Refactoring a monolithic algorithm into a class improves readability, which is is our #1 goal.
  • Lots of concrete examples in the article

Brian #5: pico-pytest

  • Oliver Bestwalter
  • Super tiny implementation of pytest core. 25 lines
  • My original hand crafted test framework was way more code than that, and not as readable.
  • This is good to look at to understand the heart of what test frameworks do
    • find test code
    • run it
    • mark any exceptions as failures
  • Of course, the bells and whistles added in the full implementation are super important, but this is the heart of what is happening.

Michael #6: An Introduction to Cython, the Secret Python Extension with Superpowers

  • Cython is one of the best kept secrets of Python.
  • It extends Python in a direction that addresses many of the shortcomings of the language and the platform, such as execution speed, GIL-free concurrency, absence of type checking and not creating an executable.
  • Number of widely used packages that are written in it, such as spaCy, uvloop, and significant parts of scikit-learn, Numpy and Pandas.
  • Cython makes use of the architectural organization of Python by translating (or 'transpiling', as it is now called) a Python file into the C equivalent of what the Python runtime would be doing, and compiling this into machine code.
  • Can sometimes avoid Python types altogether (e.g. sqrt function)
  • C arrays versus lists: Python collection types (list, dict, tuple and set) can be used as a type in cdef functions. The problem with the list structure, however, is that it leads to Python runtime interaction, and is accordingly slow
  • Nice article for getting started and motivation. But I didn’t see Python type annotations in play (they are now supported)

Extras

Brian:

Michael:

Jokes

  • What do you call eight hobbits? A hobbyte.
  • Two bytes meet. The first byte asks, 'Are you ill?' The second byte replies, 'No, just feeling a bit off.’
  • OR: What is Benoit B. Mandelbrot's middle name? Benoit B. Mandelbrot.


Audio Download

Posted on 30 May 2019 | 8:00 am


#131 Python 3 has issues (over on GitHub)

Sponsored by DigitalOcean: pythonbytes.fm/digitalocean

Brian #1: PEP 581 (Using GitHub issues for CPython) is accepted

  • PEP 581
  • The email announcing the acceptance.
  • “The migration will be a large effort, with much planning, development, and testing, and we welcome volunteers who wish to help make it a reality. I look forward to your contributions on PEP 588 and the actual work of migrating issues to GitHub.” — Barry Warsaw

Michael #2: Replace Nested Conditional with Guard Clauses

    # BAD! 
    def checkout(user):
        shipping, express = [], []
        if user is not None:
            for item in user.cart:
                if item.is_available:
                    shipping.append(item)
                    if item.express_selected:
                        express.append(item)

        return shipping, express
    # BETTER! 
    def checkout(user):
        shipping, express = [], []
        if user is None:
            return shipping, express

        for item in user.cart:
            if not item.is_available:
                continue

            shipping.append(item)
            if item.express_selected:
                express.append(item)

        return shipping, express

Brian #3: Things you’re probably not using in Python 3 – but should

  • Vinko Kodžoman
  • Some of course items:
    • f-strings
    • Pathlib (side note. pytest tmp_path fixture creates temporary directories and files with PathLib)
    • data classes
  • Some I’m warming to:
    • type hinting
  • And those I’m really glad for the reminder of:
  • enumerations
    from enum import Enum, auto
    class Monster(Enum):
        ZOMBIE = auto()
        WARRIOR = auto()
        BEAR = auto()

    print(Monster.ZOMBIE)
    # Monster.ZOMBIE
  • built in lru_cache: easy memoization with the functools.lru_cache decorator.
    @lru_cache(maxsize=512)
    def fib_memoization(number: int) -> int:
        ...
  • extended iterable unpacking
    >>> head, *body, tail = range(5)
    >>> print(head, body, tail)
    0 [1, 2, 3] 4
    >>> py, filename, *cmds = "python3.7 script.py -n 5 -l 15".split()
    >>> cmds
    ['-n', '5', '-l', '15']
    >>> first, _, third, *_ = range(10)
    >>> first, third
    (0, 2)

Michael #4: The Python Arcade Library

  • Arcade is an easy-to-learn Python library for creating 2D video games. It is ideal for people learning to program, or developers that want to code a 2D game without learning a complex framework.
  • Minesweeper games, hangman, platformer games in general.
  • Check out Sample Games Made With The Arcade Library too
  • Includes physics and other goodies
  • Based on OpenGL

Brian #5: Teaching a kid to code with Pygame Zero

  • Matt Layman
  • Scratch too far removed from coding.
  • Using Mu to simplify coding interface.
    • comes with a built in Python.
    • Pygame Zero preinstalled
  • [Pygame Zero] is intended for use in education, so that teachers can teach basic programming without needing to explain the Pygame API or write an event loop.”
  • Initial 29 line game taught:
    • naming things and variables
    • mutability and fiddling with “constants” to see the effect
    • functions and side effects
    • state and time
    • interactions and mouse events
  • Article also includes some tips on how to behave as the adult when working with kids and coding.

Michael #6: Follow up on GIL / PEP 554

  • Has the Python GIL been slain? by Anthony Shaw
  • multithreading in CPython is easy, but it’s not truly concurrent, and multiprocessing is concurrent but has a significant overhead.
  • Because Interpreter state contains the memory allocation arena, a collection of all pointers to Python objects (local and global), sub-interpreters in PEP 554 cannot access the global variables of other interpreters.
  • the way to share objects between interpreters would be to serialize them and use a form of IPC (network, disk or shared memory). All options are fairly inefficient
  • But: PEP 574 proposes a new pickle protocol (v5) which has support for allowing memory buffers to be handled separately from the rest of the pickle stream.
  • When? Pickle v5 and shared memory for multiprocessing will likely be Python 3.8 (October 2019) and sub-interpreters will be between 3.8 and 3.9.

Extras

Brian:

Michael:

Jokes

  • MK → Waiter: Would you like coffee or tea? Programmer: Yes.


Audio Download

Posted on 21 May 2019 | 8:00 am


#130 Python.exe now shipping with Windows 10

Sponsored by Datadog: pythonbytes.fm/datadog

Folks this one is light on notes since we did it live. Enjoy the show!

Special guests

Topics


Audio Download

Posted on 14 May 2019 | 8:00 am