By Audrey Roy
I’m partial but I think she did a great job of pointing out where Python might/should go.
By Jeff Rush
“Metaprogramming is the programming of programming“
Note
Apologies, but I couldn’t keep up with the firehouse of knowledge. This is woefully incomplete.
Warning
This is from me: Metaprogramming should be done carefully. Only use it when you have no other options!
- metaclasses
- decorators
- attributes
- more
Code Manipulates Data
Data Feeds into Code
Metaprogramming sits about the Code/Data
Add/adjust elements
register elements
tag elements
event management
- Pre/post initialization
- read/write
- call-and-return
class Request(object):
class HTTPServer(object):
def handle_request(self, ...):
req = Request(...)
objective
- subclass a class deep inside a module
- Requires rearrangement content of the module
old_import = sys.modules['__builtin__']
sys.models['__builtin__'].__import__ = self.__import__
class Member(object):
__metaclass__ = DatabaseTable
dbtable = "Members"
class DatabaseTable(type):
def __init__(cls, name, bases, class_dict):
col_defs = db.query_cols()
for col_def in col_defs:
db_column = wrap_col_rdwr(col_def)
setattr(cls, col_def.name, db_column)
def wrap_col_rdwr(col_def):
def get_dbcol_value(self):
return self.__dict__.get(col_def.name, None)
def set_dbcol_value(self, value):
value = col_def.validate(value)
self._-dict__[col_def.name] = value
return property(get_dbcol_value, set_dbcol_name)
The latter are much simpler
The latter can do almost everything the former can
- only a metaclass can add methods to the class itself
class-level services (methods)
- @classmethods provide them to instance
- metaclass methods provide them to the class itself
only a metaclass can add to class attrs not visible to self
- meta-methods
- meta-properties
Class decorated can be (more easily stacked)
MISSED BULLET
def CallTracer(attr):
""" Do custom logic stuff here """
return attr
def tracecalls(cls):
def my__getattribute(self, name):
attr - super(cls, self).__getattribute__(name)
return attr if not callable(attr) else CallTracer(attr)
cls.__getatttribute__ = my__getattribute
return cls
@tracecalls
class MyClass(object):
pass
The talk dived a bit into things like:
Diving into this sort of code is tricky, because the reasons for use of these tools is not necessary in 99% of Python projects. I prefer to rely on decorators to alter behavior because they are syntactical sugar. Easy to find and very explicit.
by Mark Ramm
Tools matter
- Python is a great tool
- “SQLAlchemy is the best ORM on the planet regardless of language”
Data matters just as much
Know your tools
Screws and nails
- deck (screws are stronger than nails, but harder to use)
- siding (nailing yourself to the siding)
- mongodb (nailed the SourceForge team to the wall via MongoDB)
Takeaway: If you don’t know your tools when you hit production you are going to be trying to debug something critical.
“Who here has used a non-relational database? Not many? Oh, there’s this thing called a FILESYSTEM.“
- Atomicity
- Consistency
- Independent
- Durable
I can’t use NoSQL because
- It’s financial data (need consistency)
- my data is relational
Mark says BULLSHIT
- Amazon tracks financial data using NoSQL
- You can make things more durable and consistent if you know what you are doing
DSL for ad-hock queries understood by everyone
- Business and marketing types can run their own queries
blah blah inconsistencies blah blah latency
Being down is better than being wrong
- Critical for medical applications!
Many different types:
key/value store
distributed key/value store
column oriented stores
map-reduce store/system
documented oriented store
- MongoDB
- CouchDB
- XML (now we use JSON instead of this)
- ZODB (A really good system in Python - just so long as you don’t want to share your data with other languages)
graph oriented stores
It is impossible for a distributed computer system to simultaneously provide all three of the following guarantees:
Google and Facebook scale has to handle downtown gracefully so has to choose which of these three things they’ll focus on
NoSQL systems tend to have a few but not all of the following bullets:
Mark calls BULLSHIT on WebScale
99% of sites don’t care about this issue
99% of people are okay if their sites have a feature that fails on a continent
Think about what you actually need
- Don’t implement a database just cause it’s cool
merciless.sourceforge.net
SQL version:
select * from document where x=3 and y="foo"
MongoDB version:
b.things.find({x:3, y: "foo"});
Figure out what YOUR app needs
- SQL, MongoDB and Cassandra is great at some things
- Mark couldn’t answer this for Hadoop
Don’t obsess about SCALE you’ll never achieve
Launch your project, then worry about scale