Iterators in action

Overview

  • Functional LINQ
  • Recursvie Generators

Functional LINQ

  • LIN primitives
  • Relational Algebra
  • map, reduce reduce
  • Lots of functionality
  • Language INtegrated uery
  • Streaming

Row composition

  • __iter__, next
  • All iterators use it

Column Composition

  • Colums are harder
  • Fundamental ops in relational algerbra

Naming

  • Solution to integration in python is naming
  • namespaces

namedtuple

  • Sublcassing the tuple type
  • associated the column names with an index in the tiple
  • part of 2.6 - collections.namedtuple # LOOK THIS UP!!!

Select

  • write helper functions
  • look up tee - part of itertool.tee

operator.itemgetter

  • extracts multiple items from a seuence

create join primitives

  • hash_join adding two iters on a defined predicate

Research items

  • itertools in general
  • collections.namedtuple
  • collections in general

Concepts for use

  • Joining SQl data with other data for mashups
  • Best to convert all data to namedtuples because then you have more finicky control

Recursive Generators

  • # cookbook recipe 190465
  • Graph traversal across edges and nodes
  • Workhorse algorithm
  • Bone up on yields again

Fork

  • 2.5+
  • Given a generator runs it in a separate thread
  • Look up fork

Global Interpreter Lock

  • Not in cpython or in jython
  • from __future__ import global_interpreter_lock
  • from __future__ import GIL
  • Not going to happen!