Posted by kev
Sun, 16 Oct 2005 01:51:00 GMT
Jim Weirich is awesome. An excellent speaker, a good coder and a good teacher. His slides are so well done I don’t even have notes for him.
Edit: Accidentally had Jim’s page linked wrong. Its .org. This has been fixed.
Posted in Coding, Ruby | no comments | 1 trackback
Posted by kev
Sun, 16 Oct 2005 01:35:05 GMT
So Ryan had really awesome stuff, and my computer crashed as I finshed my notes. I’ll link to him ASAP.
Posted in Coding, Ruby, Hacks | no comments | 3 trackbacks
Posted by kev
Sat, 15 Oct 2005 22:38:16 GMT
Hi guys. You may have noticed that my blog has been down or doing odd things today. This is because I was helping to test a bugfix for a ticket I turned in a while back. Good news, its fixed and will be part of the Release Candidate later tonight.
What this means: those of us on textdrive or other reverse proxied systems can use named routes without fear of the port number showing up. Huzzah or something similar.
Posted in Web Dev, Coding, Rails, Ruby | no comments | no trackbacks
Posted by kev
Sat, 15 Oct 2005 20:54:40 GMT
Kevin Baird
What is No Clergy?
Setup
- Performers read music via web browser
- Equivalent of turning written pages is a web page refresh
- The conductor is a bash script
(Sample of trumpet and violin musical score pages)
2nd and Later Pages
- The audience has access to web forms in the concert space. The audience is able to modify pitch, direction, volume, etc. for the various characteristics. They can express how strong their conviction is. “I mean it, damnit!” or “Take it with a grain of salt.”
- Web forms are specific to each instrument
I should mention that I wanted web forms because i serves a useful aesthetic and functional performance. People are familiar with web browsers, and it is easy for people to bring laptops to the concert space in order to participate.
Markovian transformations are used. A marko chain is a data set that is a derivation of some previous data set. The new data set bears a strong resemblance to the original set, but is a new set. It’s used for things like modeling a (shakespeare play?) The point of using it here is so that the new page of notation is similar to the original. That makes it not terribly interesting for the second page, but by the third page, it gets interesting.
(Sample pages of trumpet and violin musical scores with audience having asked for fewer notes, or more intensity for each instrument).
- By the second page, instruments are already starting to diverge
Originally written in Python
The piece was originally written in Python. Scores, measure, and notes are used. A sound or silence event is defined. A struct keeps the state.
Data formats & output
The output is stored in MusicXML. It renders notation output via Lilypond, which renders LaTeX to PDF or other output formats.
Why Ruby?
- Flexibilit of Perl without the kludges and ugliness
- Real access control and scope
- Ruby’s specific take on blocks and iterators
- Influence of Lisp, although not aware at the time
- Open Classes vs. Paul Graham’s On List
- Also mentioned in Agile Web Development With Rails
- Self-education
Why Refactor?
- Had started informal code. It was a mess and so I said, “Why not refactor it and do it in Ruby?”
The Changes: Ruby vs. Python
- Differences…
- Culture not just technical features
- Thanks to Hal Fulton for The Ruby Way
- Much heavier use of boolean predicates… e.g. has_something?
- Rdoc encouraged much better documentation
Substitute Algorithm
Patterns
- Merge a superclass and a subclass together
- Collapse Hierarchy moves superclasses and subclasses together. this was developed in a very ad hoc process. I tried to visualize how I would generate the initial pages vs. the subsequent pages.* Extract Interface – Move conceptually similar methods into a separate module
*The most horrible name in the project is “tupplable” which refers to a musical term.
- Extract Method – Break a larger code fragment into separate methods
- I broke a lot of methods up. In the note class, I had a notation for rendering the output. I broke it up into a public method and several privates representing the notes, pitch, etc.
- Extract Superclass / Pull up method – Move methods into a superclass
- I made a new parent class called HasSound that represents the ….
Various instruments have this thing called transposition. I used this as a class variable and many things I had used as an instrance variable became class variables, and occasionally symbolic constants.
Conclusions
Future Plans
- Continue some refactoring. Dynamics are only meaningful for notation and should be proken out in case I am not producing output for notation.
- Eliminate bash wrapper scripts, convert to pure Ruby. Thankfully, I followed Matz suggestion to learn some Lisp and SmallTack, it was useful in understanding Ruby.
- Make transformations and XML storage facilities avaliable as general libraries
- Implement transformation unrelated to audience feedback. Blend some styles.
- Python good / Ruby better (for me)
I care about encapsulation
- Good design in itself
- Practical benefits for headspace issues
- Aided by having Real Access
Ruby’s openness, flexibility and transparency made each Refactoring easier and cleaner
- Perl taught me to hate sigils
- Ruby taught me to love them again
Q & A
Speaker: I’d like to ask the audience about Packaging, inclined to package once its ready.
Q: Do you have any recordings of what this sounds like live?
A: I don’t have any with me, the audio quality isn’t great.
I found that focusing on output gives me a niche… and I love that everything you hear isn’t just blowing out of a wooden tube from a hundred years ago or something like that.
David Black: I’d say it’d be good to release as a gem. If it requires lillypond, then it requires lillypond.
Q: I’m a trumpet player, and I was wondering if there would be an option to say how much they hate a specific musician? Is there a way to make sure that the music is not too physically difficult to play (for the performers)?
A: I’ve talked to my advisior, and is it aesthetically and conceptually correct to limit what gets output to what is physically playable? What if something comes out that is logistically unplayable? Well, do your best. You’re the elected official. This will be highly dependent on your performers.
Q: Just what you said about the playability and all that… just the concept of constraints. I’ve wondered if you’ve considered adding the ability to add constraints, something like that?
A: I’m somewhat reluctant to do too much on that, because I want to give the audience a lot of freedom in an environment where they haven’t had it before. When something impossible happens, that’s when… alright make the trumpet’s a little higher and the … a little lower.
Q: You really seem to be into refactoring; have you noticed a relation between the way how you think about the structure of a software program and how you think about the structure of a piece of music?
A: That’s a really interesting question. I think I’m more inclined to do it with code… and obviously people will disagree… [with code] it’s closer to saying this is the right way to do it and this is the wrong way to do it… but with music I find it a lot more subjective.
Q2: Refactoring is, as far as I know, changing how something happens without changing what the final outcome is, whereas in music the output is different once you’ve changed something.
A2: We can argue about this later
Q: Why the name?
A: It’s not an anti-religioius… musical professionals could be seen as a type of clergy. I had an earlier pice called no cathedral. This is a logical extension of that… so how could I make it more anti-clerical than that? Get rid of the clerics. It’s a metaphor.
Q: Speaking as someone who’s performed a bit of contemporary music, I wonder what the reaction has been by the audience?
A: Generally it has been very good, but with that there is a caveat: I could only get other composers to perform this for me.
One thing I should mention about the notation is that the sound of the music is identical, but the beat marks are….So that would be an improvement in presentation. That would help in that regard.
A: I hear good things about smalltalk. Actually, last weekend I started learning about smalltalk and lisp. Actually, I
Q: How many people have used a refactoring browser before? How many people miss it in Ruby?
Even in the lobby last night, I was not changing things because it was painful to change things in more than one place. I was lazy, or I din’t have a refactoring browser. If anyone is interested, I would love to get this started. [Matt Shelleburty]
Q: This is another question about the packaging… you presented it earlier as an iether/or… why is that? (Debian/ruby).
A: I’m lazy.
Q:
A: That’s a good point, and that’s the ideal solution. Yeah, that’d be the way to go probably.
Q: That’s the way gems work, I’ve worke din Debian mostly. You might look in to (…) [Stephanie]
Q: Mine was in response to the refactoring guy’s question. Things to point out is what environment you’re working in. Things like RDT which I don’t use because it brings your machine to a crawl. I think a bunch of people started, tried to put it in FreeRIDE… I don’t know if that would be possible. If anyone is interested in putting it in Emacs.
Q: Just a clarification on that too… one thing that furstrates me on that too… theres’ so many smart people that have spread themselves out. [missed a lot here] I’d rather do code than GUI anyway.
Q: I apologize in advance to continuing to steal your thunder on this refactoring thing. But since the topic came up. I’m one of the founders of the FreeRide IDE project, along with Rich Kilmer. I’m very interested in getting refactoring support in FreeRIDe. []’s got problems. Enough problems that we actually removed it from FreeRide. If there’s people that are interesteed in working on it and bringin it up to snuff, I’d be really interested in talking about it. Thanks.
Q: How big of a role did Unit Testing play in refactoring? And second, I didn’t understand why you ditched PYthon.
A: With regard to the changeover, I knew I wanted to do a substantial re-write. I was learning Ruby at the time, and everything I learned about it I thought ‘why did I learn Python’? Fortuitous coincidence in time I guess.
Posted in Coding, Ruby | 1 comment | no trackbacks
Posted by kev
Sat, 15 Oct 2005 02:54:00 GMT
Eric Hodel
Once upon a time…
- Eric and Ryan were hacking some Ruby related C
- And it sucked
*
MetaRuby
- Will implement Ruby in Ruby
- Core libarireis, parser, interpeter
MetaRuby Architecture
Why?
- Writing Ruby internals in C requries mental context switch every time you change bwetween RUby and C
Example of C code vs Ruby code.
- More Familiar
- More approachable
- Less to do
- No NULL termination
- No tainting or freezing
Inspirationsal Projects
- Sqeak Smalltalk
- Self
- Pascal, Modula-2, Oberon by Wirth
- All of these are written in themselves
Related Projects
- Matju’s MetaRuby
- YARV
- JRuby
Matju’s MetaRuby
- Different goal much more complex
- Abstracted core classes
YARV
- Ruby interpreter replacement
Rubidium
- Ruby interpreter replacement
- Rubidium is an optimizing Ruby interpreter
Rubytests
- Unit tests for Ruby
- Not comprehensive enough for our goals
- Not much work making it more complex
JRuby
- A 1.8.2 compatible Ruby interpreter
- Most builtin Ruby classes provided
- Support for interfacing and defining Java classes in Ruby
- Uses Rubytests
Current Work
Methodology
- Generate a stubbed class to overlay
- Drive unit tests to failure
- Identify core methods (primitives) that have to exist
- Fix bad tests that pass despite no implementation
- Drive all tests to green
Passing Tests
- TrueClass, FalseClass and NilClass
- Time
- Range
- NilClass
- Array
- String
Overlaid Classes
These classes overlay their core classes using Ruby’s C allocation and initialization methods replacing as many methods as possible
- TrueClass
- NilClass
- Array
- String
Replaced Classes
Rubytests
- Stale
- Mostly tests Ruby 1.6 language features
- Low test coverage
- Not fully converted to Test::Unit
- Way too much code from pre-testunit
Test::Unit
- Needs lots of methods to work
- Too complicated to refactor
- Working on core classes is hard
Future Work
Primitives
- Will be automatically translated to C
- What is a primitive?
- Implement as much as possible in Ruby
- Whatever is left becomes a primitive
** Unless we can break it down
- Choosing primitives is a discovery process
Ruby2c Translation
- Ryan will cover this a lot more
- Only necessary for primitives
Memory Allocation (Objects)
- Currently Array and String sit on top of C Ruby
- Write object allocation in pure Ruby using current memory system for all objects
- Then we will replace the memory system with a pure Ruby system
Replace core ruby library
- Works!
- Well.. kind of..
- Compiles
- Links!
- Segfaults!
- Needs alot of ping pong
Far Future Work
h5. The Groveling Commences
Parser
- Ripper is our best target
- Almost entirely Ruby already
- Just one file is in C, which we can rewrite
Object System & Garbage Collector
- Steal ideas from Sqeuak Smalltalk, Self, current Ruby
- In theory it should be easy to do
- In reality it will be hard to do well
- We’d love someone to work on this
Interpreter
- YARV or eval.c (Ruby 1.8)?
- Rubidium?
- Needs to we written in Ruby
- We’d love someone to work on this
C Extensions & C Standard Library
- Why are you writing pure C anyways?
- Probably need Ruby/C compatability stubs
- Will need to follow current Ruby/C naming conventions
Array#fill
Eight ways to call
- array.fill(obj)
- array.fill(obj, start[, length])
- array.fill(obj.range)
- array.fill {|index| block }
- array.fill(start…
“foo”.sub(/f(o)o/) { $1 }
- $1 is a “magick” read only global
- $1 can’t be set from pure Ruby
- So the interpreter needs to help us out
- Applies to all match variables
String#split
Time.rb Needs Metal
- Easy
- the_time.month
- thetime.tof
- etc
- Hard
- Time.now requries calling libc’s gettime method
- Currently we have libcwrap.rb that uses RubyInline to call into C funcitons
2 comments | 2 trackbacks
Posted by kev
Sat, 15 Oct 2005 02:29:00 GMT
Koichi SASADA
Caution! (review)
- I can’t speak English well
- If I say strange English, you can see the slide page
** Or ask another Japanese. They can speak English well.
- If you have any queestions, ask me with:
** Japanese (recommended)
** Ruby, C, Scheme, Java, …
** IRC (#rubyconf on freenode)
Agenda
- Self Introduction and Japanese Activities
- Overview of YARV
- Goal of YARV
- Current YARV status
- YARV Design, Optimization Review
- Evaluation
- Conclusion
Self Introduction
- “SASADA” the family name
- “Koichi” is given name -> “ko1”
- A Student for Ph.D. 2nd grade (Not a Son-shi)
- Systems Software for Multithreaded Arch
** SMT/CMP or other tech
** i.e.: Hypter threeading (Intel), CMT (Sun), Power (IBM)
** OS, Library, Compiler and Interpreter
** YARV is my first step for parallel interpreter
- Computer Architecture for Next Generation at Public Position
- Nihon Ruby no Kai
- Organized by Mr. Takahashi (maki)
- “Rubyist Magazine”:http://jp.rubyist.net/magazine
- vol 10 at 10 Oct 2005
- 1st anniversary at 6 Sep 2005 (vol 9)
- Ruby-dev summary
- English Diary some days
Our Activity: Rubyist Magazine
- Many Japanese articles related to Ruby
- Cooperate with Ruby Code & Style?
- I’m writing YARV internal named “YARV Maniacs”
- Many interviews of Japanese Rubyists
RubyMa!
- Published 1 Apr 2005 (April Fools)
- Parody of Negima!
- Many useful articles
- The Takahashi method:
def Takahashi
end
Overview of YARV
Overview: Background
Overview: FAQ (review of last year FAQ)
- Q: How is “YARV” pronounced?
- A: You can pronounce “YARV” what you like.
- Q: Should I remember the name “YARV”?
- A: No. If YARV succeeds, it gets renamed to Rite, if it doesn’t, no one will remember it
- About YARV, name is NOT ???
Overview: YARV System
Ruby Program –> Compiler –> YARV Instruction Sequence –>
==> Virtual Machine
==> AOT Compiler –> C Source –>
Overview: Current Interpreter
- Ruby Program: a = b + c
- Syntax tree: (a =) -> (method dispatch + (b), (c))
- Current interpreter traverses AST directly
Overview YARV - Stack Machine
The Goal of YARV
- YARV: Yet Another RubyVM -> The RubyVM
- To be the Ruby 2.0 VM Rite
- Fastest Ruby Interpreter
- Easy to be the current Ruby interpreter
The Goal of YARV (cont.)
- Support all Ruby features
- Include Ruby 2.0 new syntaxes
- Native thread support
- Concurrent execution (Giant VM lock)
- Parallel execution on parallel machine
- Multi-VM instance
- Same as Mutlti-VM in Java
Goal: Ruby 2.0 syntax
- Matz will decide it :-)
- ”{|…| …}” == “->(…){ … }”
- “I think this is ugly” – Ko1
- Multiple-values
- Same as Array? Or first class multiple-values support?
- Selector-namespace?
Goal: Native Thread Support
- Three different thread models
- Model 1: User-level thread (green)
- same as current Rubt interpreter
- Model 2: Native thread with giant VM lock
- Same as current Ruby interpreter
- Easy to implement
- Model 3: Native-thread with fine grain lock
- Run ruby threads in parallel
- For enterprise?
Goal: Native Thread Support (cont.)
h5. Current Ruby Interpreter & Model 1
- CPU1: Thread 1 -> Thread 2 -> Thread 1
- CPU2: Idle……..
h5. Model 2: Native thread with Giant VM Lock
- CPU1: Thread 1 -> (Lock) -> (in OS thread 2) Thread 2 -> (Lock) -> Thread 1
- CPU2: Idle……..
On this system, other threads can run (but the Ruby threads switch cpus with a lock)
h5. Model 3: Native thread with Fine Grain Lock
- CPU1: Thread 1……
- CPU2: Thread 2……
Goal: Native Thread Support Summary
|. |. Model 1|. Model 2|. Model 3|
|Scalability|Bad|Bad?|Best
|Lock overhead|No|Some|High|
|Impl. Difficulty|Norm|Easy|Hard|
|Portability|Good|Bad|Bad|
Goal: Multi-VM Instance
Multi-VM Instance + Thread Model 2
CPU1: Thread 1 -> (Lock of VM1) -> Thread 2 -> Lock of VM1
Goal: Load Map
- All Ruby features support
- Native Thread Support
- Experimental: Dec. 2005
- Complete: 2006?(model 2) 2007?(model 3)
- Multi-VM support
- Experimental Feb 2006
- Complete: 2006?
Current Status of YARV
Status: System
Some almosts, an incomplete and a not yet
Status: Supported Ruby Features
- Almost all Ruby features
- Not supported:
- Few syntaxes … {|*arg| …}
- Visibility
- Safe level ($SAFE)
- Some methods written in C for current Ruby implementation
- Around Signal
- C extension libraries
** (Because YARV can’t run mkmf.rb)
Status: Versions
- 0.2 YARV as C Extension
- Need a patch to Ruby interpeter
- 0.3 (2005-8): YARV as Ruby Interpreter
- merged to Ruby source (1.9 HEAD)
- Maintained on my subversion repository
- Latest version: 0.2
- Native thread (pthread / Win32) supports model 2
YARV 0.2.x
(Ruby Interpreter (Evaluator)) -> YARV (Compiler, VM, Optimizer) -> back
YARV 0.3.x
“Status: Compile & Disasm CGI”:http://www.atdot.net/yc/
Status: VM Design
- 5 registers
- PC: Program Counter
- SP: Stack Pointer
- CFP: Controler Frame Pointer
- LFP: LOcal frame pointer
- DFP: Dynamic Frame Pointer
- Some stack frame
- Control stack and value stack
Status: Optimization
- Simple Stack Virtual Machine
- Re-design Exception handling
- Peep-hole optimization on compile time
- I gave up static program analysis
- Dynamicity is your friend but my ENEMY!
- Direct Threaded code with GCC
- Specialized Instruction
- i.e. Ruby program “x+y” compiled to special instruction instead of a method dispatch instruction
- In-line Cache
- In-line Method Cache
- In-line constant value cache
** Because ruby’s “constant variable” is not constant!
- Embed values in an instruction sequence
- Unified Instruction
- Operands Unification
- *InsnA x -> InsnA_x
- Unified instructions are auto generated by VM gen
- I only decide which instructions should be combined
- Stack Caching
- JIT Compilation
- I made easy one for x86, but…
- Too hard to do alone. I retired.
- AOT Compilation
- YARV bytecode -> C Source
- Easy to develop
- Hard to support exception
Status: Demo
- YARV building demo?
- YARV running demo?
Status: Evaluation
- Yield block is not fast (2-3 times faster than C Ruby) - optimizing this will be work for the future
- With all optimizations, basic math can see a 50 times performance gain over C Ruby
- Ackermann can see 20 times gain over C Ruby
- Wow - YARV as it stands stacks up really well against other interpreters for basic math type stuff
Status: Awards
- 2004: Funded by IPA Exploratory Software Development “Youth”
- IPA: Information-technology Promotion Agency, Japan
- 2005: Funded by IPA Exploratory Software Development (continuance)
- 2004: got awarded “Super creator” from IPA
Conclusion
- YARV supports almost all Ruby syntax
- YARV suppoorts some RUby libraries
- YARV 0.3.2 supports native thread
- YARV achieves significant speedup for ruby programs which have VM bottleneck
- This means that we can enjoy Symbol programming with Ruby
Conclusion: Future Work
- Support all Ruby features
- Support every thread model
- Support multi-VM Instance
How Can You Help me
- Any comments are welcome
- Build reports, Bug reports, architecture reports, …
- yarv-devel Mailing List
- English ML for YARV development
** Matz and other Japanese join
- “YARV Wiki”:http://yarv.rubyforge.org/pukiwki/pukiwiki.php
- Give me a job! (I’ll finish my course 2 years later)
Special Thanks
- Matz the architect of Ruby
- IPA: His sponsor
- YARV development ML subs
- All rubyists
Q&A
Posted in Coding, Ruby | 1 comment | no trackbacks
Posted by kev
Sat, 15 Oct 2005 01:24:36 GMT
jruby.sourceforge.net
Charles Oliver Nutter, presenting
Who Am I?
- Charles Oliver Nutter: headius@headius.com
- Senior Architect/Technologist at Ventura Corp
Part1: Past and Present
- What is JRuby?
- Why JRuby?
- Peer Comparison
- The Real World
What is JRuby?
- A “100% Java” based Ruby interpreter
- Mostly 1.8 compatable
- Four years and 15 developers; currently 3-5 active and under heavy development
- Originally modeled on Ruby 1.6 code
- Tri-licensed: CPL, GPL, LGPL
- Sun J2SE 1.4 or higher (FOSS Javas in future)
- Java/Ruby integration getting better
Why JRuby?
- JVM provides native threading, generational GC, and extensive networking and database support
- Wealth of libraries and frameworks; large userbase, wide deployment
- Many Javaists would like to use Ruby more
- Java is “just another platform” for Ruby
- JRuby could help grow Ruby the language apart from C Ruby
- Sun, others very interested in dynamic (typed) languages on the JVM
- Javaists (by choice or by force) can help promote Ruby
- Java/Ruby integration merges best of both
- Ruby + J2EE = enterprise ruby that managers can swallow
- Ruby + J2ME… someday?
Demo 1: Java integration
- Ruby code mixed into java
- JDBC used for DB access (PostgreSQL 8.0)
- Mostly transparent object marshalling
Peers
The Real World
- RDT: A Ruby IDE for Eclipse; rubyeclipse.sourceforge.net
- jEdit: A Multi-Language Code Editor; www.jedit.org
- DataVision: Java-based Reporting Software; datavision.sourceforge.net
- Internal projects
- Need more
How “Ruby” is it?
- Of 1049 Rubicon tests, 80% succeed
- Temporary incompatibilities
- Ruby thread semantics differ from Java’s
- No continuations
- Twice as slow (half as fast?) as C or worse
- YAML: no up-to-date, working pure Ruby or Java parsers
- still missing a few 1.8 methods
- Permanent incompatibilities
- System calls, C-language Ruby extensions, anything to do with C
- Platform-specifics: file stats, permissions, process launching, signals
Part Two: The Future
- Continuing to improve compatibility
- Running mainstream Ruby apps
- Improving Java integration
- Speeding up
- The New JRuby
What needs to change?
h4. What needs to change?
- JRuby deficiencies ( as of 0.8.2)
- Stack depth (~ fib(280))
- Threading and thread semantics
- Continuations support
- Speed
- Consistency, maintainability
- Compilation
- Better use of Java’s strengths
- Tigher integration between Java and Ruby
- Ruby deficienies (as of 1.8):
- Stack depth (~ fib(1325))
- Native threading
- Speed
- Compilation
The New JRuby
- Stackless; Continuation Passing Style (roughly)
- Iterative interpreter
- min threading model
- Compilation to Java bytecodes, offline and JIT
- Pluggable architecture
- Seamless, powerful Ruby/Java integration
- Behave in controlled environments
- FAST
Milestones and progress
- Stackless, iterative proof of concept (POC) (Sept 15, complete)
- Redesign, refactoring of POC (Oct)
- Reimpl of interpreter based on POC (Nov)
- Reimpl of built-in classes (Nov-Jan)
- Threading engine (Jan)
- Tri-call optimizations (Jan)
- Continuations (Jan)
- Compilation (Feb - Apr)
- Complete for JavaONe 2006
Demo 2: Fibonacci
- Recursive fib algorithm (contrived, I know)
- Jruby 0.8.2: shallow
- Ruby: deeper
- JRuby “stackless” POC: deepest
(Demo of doing fib 30000 in JRuby! Pretty cool.)
(Somewhat longer demo of 150000. Also cool.)
What Else?
- YARV bytecode execution
- MetaRuby’s “Ruby in Ruby” useful to JRuby
- drb proxy to RMI
- ActiveRecord JDBC connector
- WEBrick-mimicking servlets
- Other ideas?
Part Three: What now?
- Redesign is in full swing
- Heavy refactoring of JRuby core
- A better Ruby than ruby?
- Help Wanted!
- zlib implementation using Ruby-Java integration
- FIle locking using Java’s NIO (New I/O)
- Feature-complete YAML support
- Running mainstream Ruby apps, isolating and reporting errors
- Help with new design and with refactoring effort
- Tangibles
Q&A
- “JRuby”:http://jruby.sourceforge.net
- SF project page”:http://sourceforge.net/projects/jruby
- JRuby mailing lists on SF
- Charles Oliver Nutter: headius@headius.com
- Thanks to:
- Thomas ENebo: JRuby project manager
- Kelly Nawrocke: Jruby developer
- David Corbin: JRuby developer, RDT developert
- Special thanks to Jan Ame Petersen, original JRubyist
REAL Q/A
???: About YAML - parser written in C, have C to Java translators been tried?
- Charles: Might not produce code that would wire in nicely; focusing in pure Ruby implementation.
David Black: What about things that it would be nice if they were different than they currently exist in C Ruby - for instance, similar behavior often goes through different code paths? Can you change those things? Will it make it less Ruby?
- Charles: Mainly taken perspective that we are following what Ruby does and following what Matz and company do. Having this other platform will point out inconsistencies; some things are unfollowable. Having two places where behavior is implemented shows inconsistencies.
???: Is JRuby going to be reentrant? Will you be able to run multiple JRuby instances in the same process?
- Charles: Yes, unable to control where calls are coming from, so needs to be re-entrant. Either that or able to run multiple lightweight interpreters in the same VM and then manage state. Not thread safe at this point but hopefully that will change.
Duane Johnson: In the demo, the each iterator isn’t acting very Ruby-like.
- Charles: The demo is kind of put together to show everything. What would probably be better would be a Ruby-Java layer that does “rubyfication.” Code as demo’d was more javaish but still simpler than real Java.
Posted in Coding, Ruby | no comments | no trackbacks
Posted by kev
Fri, 14 Oct 2005 21:55:00 GMT
Presented by Tanaka Akira
akr at m17n dot org
This one was really really fast.. here’s what I got… – Kev
Table of Contents
- Who am I?
- How to user open-uri
- Why open-uri?
- open-uri and net/http
- How to design easy-to-use api
Who am I
Who am I (1)
The author of open-uri and several standard libraries:
open-uri.rb, pathname.rb, time.rb, pp.rb, prettyprint.rb, resolv.rv, resolv-replace.rb, tsort.rb
Who am I (2)
Contribution for various classes and methods
- IO without stdio
- IO#read and readpartial
- Time Time.utc, Time@utc_offset
- allocate marsha1dump marsha1load
- Regexp#top_s Regexp.union
- Process.daemon
- fork kills all other threads
Who am I (3)
I report many bugs, over 100/year
- core dump
- test failure
- build problem
- mismatch between doc. and imp.
- etc
Who am I (4)
I wrote several non-standard libraries.
How to Use open-uri
Simple Usage
require open-uri
open(“http://www.ruby-lang.org”) {
|f|
print f.read
}
Similar to open files
Why Open-uri
- Easy to use api
- VFS: not only http
open-uri and net/http
net/http has too many ways
- Net::HTTP.get_print
- Net::HTTP.ge
- Net::HTTP.start {|h| h.get} etc
confuses users
open-uri has Fewer ways
open(uri) {|f| }
uri.open {|f| }
uri.read
Save user’s memory
reuse users knowledge
net/http: get and print
Net::HTTP.get_print( URI(“http://host”))
print Net::HTTP.get(URI(“http://host”))
open-uri: get and print
open(“http://host”) {|f|
print f.read
}
print URI(“http://host”)…
get and print
net/http
- Net::HTTP.get_print print only
- Net::HTTP.get: good
open uri
…
Why Easy?
open(“http://host)
- No new construct
- Users don’t need to learn
open-uri respects user knowledge
net/http: headers
Net::HTTP.start(“host”) {|h|
r = h.get
….
}
- No URI anymore
- No Net::HTTP.get anymore
- Net::HTTP.start, net.. and body used instead
open-uri headers
- Still URI
- still open method
- Easy to use
net/http: SSL
- Different library: net/https
- Net:HTTP.new nad Net:HTTP.start
- Different port
- Server verification…
open-uri: SSL
- Still URI
- Still open method
- Server verification by default
- No new library
- No new methods, few things to learn
net/http: proxy
- New method: Net::HTTP::Proxy
open-uri: proxy
% http_proxy = http://blah
% export http_proxy
- Conventional environemtn variable supported
- No new methods. An user might know this already
- Fewer things to learn
net/http: basic auth
- New class: Net::HTTP::Get
- New method: Net::HTTP#request
open-uri: basic auth
- Still URI
- Still open method
- New option: :httpbasicauthentication
- No new methods, few things to learn
How to design Easy-to-Use API
- Save brain power
- Evolve gradually
Fewer Things to Learn
- Fewer constructs for pragmatic usages
- Huffman coding
- DRY
- No configuration is good ocnfiguration
- Reuse user knowledge
- Infrastructure friendly
Fewer Constructs for Pragmatic Usages
*Open vs Net::HTTP.get, Net::HTTP#get etc
- This is not minimalism
- The target of “fewer” is not all constructs
Pragmatic usages should be supported by small constructs.
Fewer Constructs(2)
Diagram.. link later hopefully
Freqently use convenience methods, rarely use many primatives
Ex. nethttp and open-uri
Methods frequently used:
net/http: Net::HTTP.start, Net::HTTP#get
open-uri open
open-uri’s fewer constructs supports many more features
Huffman Coding
- Shorter for freqeuent things
- Longer for rare things
Optimize for frequent things.
Ex: p
So longer methods for rarely used things, shorter methods for frequently used things
Ex p
p obj
- Very frequently used
- Bad name in common sense
- Almost no problem because everyone knows
Ex. pp and y
- Bad name in common sense
- Problematic than
p because not everyone knows
Ex. tos and tostr
- to_s shorter. frequently used.
- to_str longer, used internally
Ex. def
- def shorter, frequently used
- define_method longer. not encouraged
Ex time.rb
- Time.parse frequently used
- Time.strptime generic, needs to learn the format.
- Time.parse is less flexible but enough for most cases, and easy to learn
Candidates for Huffman Coding
- Method name
- Other name
- Convenience method
- Language syntax
- etc
Length of Huffman Coding
- Number of characters
- Number of nodes in AST
- Editor keystrokes
- etc
Encourage Good Style
- Programmers like short code
- Short code should be designed as good style
DRY – Don’t repeat yourself
Violations are common
No Configuration is Good Configuration
Things should work well out of box.
- SSL CA certs
- http_proxy environment variable
Bad Examples
- ext/iconv/config.charset
- soapuseproxy
- require “irb/completion”
- RUBYOPT=rubygems
Reuse User Knowledge
oepn-uri reuse user knowledge
- open is used to access an external resource
Reusable Knowledge
- Ruby builtin (popular) metho
- consistency
- Unix
- Standards: POSIX, RFC, etc
- Metaphor
Consitency
- bang methods
- eachwithindex
- etc
Consistency violation:
Metaphor
- HTTP is a kind of network file system
- oepn-uri doesn’t support beyond file system: POST, etc
Infrastructure Friendly
- emacs, vi
- line oriented tools
- shell and file system
- web browser
Prefer
“It is easy using the legacy tool XXX” over “It is easy using the new tool YYY”
Evolve Gradually
- Adaptive Huffman coding
- How to find bad API
- How to avoid incompatability
Adaptive Huffman Coding
What methods are used frequently?
- Long method name at first
- Alias to short name later
- Define convenience methods for idioms
Adaptive Huffman Coding(2)
- Short names and operators should be used carefully
- Use a long name if hesitate
- Alias is not a bad thing (TMTOWTDI)
- Primitives should have long names
- Define new method for idiom
Operators
- CGI#[] and CGI#params
- CGI was defined unsuitably.
- Hash #[]
How to find bad api
- Repeated surprise
- Often cannot remember
Repeated Surprise
Example
- Time#utc is destructive
- Iconv.iconv returns an array
Often Cannnot Remember
Manual is required again and again for same issue.
Idiom
- Repeated code
- Violate DRY
- An idiom may be good or bad
Bad idiom example
How to Avoid Incompatibility
Extension without Incompatibility:
- New method
- New keyword argument
- con contants
Introducing new names has no compatability problem (in most case)
Incompatible Change is a Bad Thing
But fixing bad API…
Incompatible Change
API Migration Example
- net/http: API version
- cgi: special implementation for transition period
net/http API version
Net::HTTP has two APIs
- Ruby 1.6 API version 1.1
- Ruby 1.8 API version 1.2
net.http: switch API version
- It tens to forget restore API version
- Global switch, not thread save
cgi: special implementation for a transition period
CGI#[] returns
- Ruby 1.6 an array of params
- Ruby 1.8: Transition period
- future? : a first parameter or nil
CGI#[] returns something tweaked on 1.8
Try to work as both Array and String
- Ruby 1.8.0 subclass of String
- Ruby 1.8.1 subclass of DelegateClass(String)
- Ruby 1.8.2 …
fork: Warning after change
Does fork kill other threads in child process?####
- Ruby 1.6: No
- Ruby 1.8: Yes
fork: warning after change
- Ruby 1.6: No warning
- Ruby 1.8.0: No warning
- Ruby: 1.8.1: warning: fork terminates thread
- Ruby: 1.8.2: No warning
IO#read: warning before change
IO#read will block even if O_NONBLOCK is set
- Ruby 1.8: doesn’t block
Ruby 1.9: does block
Ruby 1.8.2: No warning
- Ruby 1.8.3: Warning
- Ruby 1.9 : No warning
Easy-to-Use vs Security
Easy to Use vs Sec
- HTTP_PROXY
- http://user:pass@host/
- redirection and taint
- File.open(uri)
VFS: Virtual File System
Why VFS?
Typical simple program
- Load na external resource
- Process the resource
- Store the result
VFS ease the first step.
What is VFS
VFS provides
- open a http/ftp resource
- read a resource
…
…
VFS and polymorphism
The polymorphism can be implemented by
- usual method dispatch calls
….
Polymorphic open
If open-uri is in effect
*open(“http://…”) calls URI(“http://…”).open
Any URI can be opened if the URI has open method
Other Resources
LDAP
Other Operations
- URI().read
- Other operations should be defined for polymorphic to Pathname future
Sec Considerations
- open(“|…”)
- File.open is not affected
Summary
- How to design Easy-To_Use API
- Save brain power
- Evolve gradually
- VFS by open-uri
Q/A
Some guy writing a book: Should I teach Array.push or Array<
Response: I think the api should lean towards teaching.
DHH: Are you going to do what you’re reading for writing?
Response: POST should be supproted in the future, but write.. eh.. not as useful.
AC: Warning would be more useful -not- at runtime..
Response: Inaudible
Posted in Coding, Ruby | no comments | no trackbacks
Posted by kev
Fri, 14 Oct 2005 20:56:00 GMT
Presented by Francis Hwang
http://fhwang.net/top_to_bottom/
First Toy Example fibonacci.rb
Testing is harder in the real world
- Complexity
- External components
- Side-effects
- You don’t want to send a thousand emails to test
- Speed
- You don’t want to drop and rebuild your schema for every single test
The Quality Elbow
- You initially get lots of quality for your cost
- As it goes on, you get lest quality from your code
- Similarly, testing degrades as code does
Message
- Simple mailer.
- Just sends stuff down the pipe
For testing we don’t want to send real emails
MockMailer
- Same methods, but its really really stupid
- Adds emails to emails sent
We can use this to simply check our mailers.
This is too coupled however, so a message ought to automatically know what mailer to use in tests.
So we use a user, and pass in the mock mailer to the user. This still isn’t right, as its not the user’s job.
We need to bite the bullet and imitate a global variable. We’ll use contextual service.
If we need to use a mock with several outside services, we can set those up in setup.
The issue with mocking a sql query is that you’re basically writing a full sql parser.
Back to slides…
Tradeoffs of Complex Mock classes
- Upsides
- Speed
- Incredible speed increase
- Not spending 3 hours to run sql tests
- No side-effects and no cleanup
- Downsides
- Indirection
- Possible bugs in mock class
- Time spent to build mock
You could mock anything!
Dynamicity is your friend
Overriding
- May not be a good idea
- Undefines FileUtil, and Dir and File
- Also require
Further improvement
- Test-centric libraries
- Try to include mocks in your libraries
- Domain-specific test languages
Questions/Comments
DHH: Use transactions and still use the db. Saves speed. Run test cases in a transaction, rollback. Basecamp tests using db does 410 tests, runs in 60 seconds.
Austin: In 3rd code example, ContextualService, isn’t that just another way of doing dependency injection (needle, etc).
Response: Yup. I like the fact that the mailer manages its own instance.
DHH: At 0.10, Rails on Needle? Decided its not an approach, what you really want is dependency injection, without knowing anything about dependency injection. What you’re really interested in is referencing the class mailer and getting the mock back when you’re running tests. You can preload the load path so when you’re using tests it uses the mock directory before the real thing. The mock can then just be called Mailer.
Azlak: One downside is writing them.. libraries can generate mock objects dynamically.
Response: YMMV, mocking is a thing mostly on the logick of the thing, so abstracting out doesn’t work well. Most mock libraries work like message interception, but its like a black box. I personally like just writing them better.
Azlak: Mixing up mocks and stubbs, whats the difference? Mocks are actually black boxes that pretend to be something else. Rails has stubbs not mocks.
DHH: You can choose to override or not. Or turn into a whole black box if you like. You can use either mocks or stubbs.
Azlak: The major difference is that if you don’t specifically say its a mock, it blows up in your face.
Blah blah blah..
AC: If you want a mockfs, why don’t you just use a specific path as a sandbox.
Response: I have classes where they know the specific paths, so you have to intercept everywhere and it gets messy.
AC: If you’re hitting a real file system you have the potential to bork your tests in the future if things crash.
Response: You may also want to check if the hd is full.. etc.
AC: Can run mock tests without actually having the server running (mysql etc).
Response: Also nice to be able to change your structure without touching a database. Also, in theory, you could use tests to test if you’re working on code running on different machines…
Dom Tersen: Mock testing are good for tests, but they actually don’t proove the system works for real. Mock object testing doesn’t replace real testing.
Patrick: Who’s responsibility is it to write the mocks? Should we write them on the side, or should the capability to be mocked be part of every IO library?
Response: I’d be happy if library authors were writing mocks for their libraries. Those writing the library are usually in the best place to write Mocks.
I’ll clean this article up more as I have time. – Kev
Posted in Coding, Ruby | no comments | no trackbacks
Posted by kev
Fri, 14 Oct 2005 20:55:58 GMT
Ok, I’m sitting in the basement of the RubyConf hotel. I’ll be giving you play by play as I can. Things should start in a few minutes.
Posted in Web Dev, Coding, Rails, Ruby | no comments | no trackbacks
