Micro or Short?

December 3, 2009

I’ve been getting some questions lately about what the difference is between generating a “micro” or generating a “short” stack is. So here it is:

Micro: Generates a structure that’s good to use with passenger out of the box
Short: Generates a structure that’s gemmable with Jeweler

They’re both Pancake Short Stacks. A micro just doesn’t have all the files / directories generated. Add them as needed and they’ll work.

Here’s the structures:

∴ pancake-gen micro foo
#snip
∴ tree foo
foo
|-- Rakefile
|-- config.ru
|-- foo.rb
|-- pancake_init.rb
|-- public
|-- tmp
`-- views
    `-- root.html.haml

And for a Short Stack:

∴ pancake-gen short bar
#snip
∴ tree bar
bar
|-- LICENSE
|-- README
|-- Rakefile
|-- VERSION
|-- lib
|   |-- bar
|   |   |-- bar.rb
|   |   |-- config
|   |   |   |-- config.rb
|   |   |   `-- environments
|   |   |       |-- development.rb
|   |   |       |-- production.rb
|   |   |       `-- staging.rb
|   |   |-- config.ru
|   |   |-- models
|   |   |-- mounts
|   |   |-- public
|   |   |-- tasks
|   |   |   `-- bar.rake
|   |   |-- tmp
|   |   `-- views
|   |       `-- root.html.haml
|   `-- bar.rb
|-- pancake_init.rb
`-- spec
    |-- bar_spec.rb
    `-- spec_helper.rb

The “micro” stack has a stack root in “foo/” and the “short” stack has a root at “bar/lib/bar”

Inherited Templates in Pancake

November 28, 2009

Update: Fixed some of the code examples to remove a stray block

Pancake uses template inheritance to embed content in a root template. This is similar to Rails layouts, but with inherited templates, you can have different content blocks, each having their own default content. Inspiration for this feature has come from Django and Rango which also use an inheritance concept. There’s really not an easy way to explain in words so lets see what it looks like:

# base.html.haml

!!!
%html
  %body
    %h1 Welcome to base
    - content_block :content do 
      %p Default Base Content

# foo.html.haml

- inherits_from :base
  
- content_block :content do 
  %p Foo Content

If we had a stack that looked like this:

class FooStack < Pancake::Stacks::Short
  add_root(__FILE__)
  
  get "/" do 
    render :base
  end

  get "/foo" do
    render :foo
  end
end

When we visit the “/” url we’d get output like this:


<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
  <body>
    <h1>Welcome to base</h1>
    <p> Default Base Content </p>
  </body>
</html>

When we visit “/foo” we’d get something like this:


<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
  <body>
    <h1>Welcome to base</h1>
    <p> Foo Content </p>
  </body>
</html>

See how the content block for :content in foo was used when we inherited from :base? It’s got “Foo Content” in the inherited one. Templates in a Pancake Short stack can all use inherited templates. By suppling inherited_from with a template name, the current template will inherit from it. You can supply a different template name on each request, and then inherit from a different template each time if you need to.

That’s cool and all. We can inherit from a different template each time. Pancake will take care of finding the right template when you inherit a stack also, so that if a child stack hasn’t defined it, it will look in the parent stack.

Wouldn’t it be cool to be able to append to the parent content rather than just overwrite it though? Well a Pancake Short stack lets us do that too.

# foo.html.haml

- inherits_from :base
  
- content_block :content do 
  %p Foo Content
  = content_block.super

By calling super on the content block, you’re inserting the content of the parent block inside the child. The output of running that template would look like this:


<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
  <body>
    <h1>Welcome to base</h1>
    <p> Foo Content </p>
    <p> Default Base Content </p>
  </body>
</html>

One of the troubles with an inherited approach though, is that if I have 2 stacks, Foo and, Bar inside a master container stack MyMasterStack, all three of these will inherit from different places. I’ve just added in Pancake version 0.1.26 the ability to specify a stack name and template name in inherits from. So you could have something like this:


- inherits_from UiStack, :base

- content_block :content do 
  %p Some content

So we’re inheriting this template from the :base template of the UiStack. Kinda cool, but there’s still a problem. I need prior knowledge that it’s the UiStack that I want to inherit from. That’s ok when it’s your app, but if you write a stack for general consumption you may not be able to make that assumption. Pancake sorts this for you. You can tell Pancake, at a global level, which Stack you’d like to set as the master template stack. By default this is the container stack when using pancake stand alone. If you’re just adding it somewhere else though, you’d start the container stack like this:

MyStack.stackup(:master => true)

This sets the stack as the master which adds the “master” directory as another root for the stack, but also sets the stack to be master of templates. To manually set the stack as master of templates you do this:


  MyStack.before_build_stack do 
    Pancake.master_templates = UiStack
  end

This sets the UiStack as the master for the whole pancake graph, which lets us do this:


- inherits_from :default!

- content_block :content do
  %p content for this view

Using the :default! template name, will tell pancake to inherit from the master template stack, in this case UiStack, with the :base template. You can tell it to use a different one but by default, :base is the one that gets used.

By using this mechanism, a collection of different shared stacks can share the same look and feel by inheriting from the global Pancake master templates stack. When writing plugins and shared stacks, you can just set the stack templates to inherit from :default! and everything should just work.

Mounted Web Apps Sites

November 23, 2009

This weekend I got to go to Railscamp 6. Railscamp is so much fun. There’s heaps of really cool ruby peeps there. I’d start to name them but there were over 120 people at this one so it may not make for the most interesting read to those who weren’t there.

The reason I’m writing this post though is not about Railscamp itself, but rather the project that I helped hack on. Lincoln Stoll flew all the way from Germany to be at Railscamp. I love catching up with him and as usual, he had plenty of interesting stuff happening. One that really caught my attention was his idea of having a proxy through to CouchDB. We’d love to be able to use CouchDB in applications with some authentication and whilst the Couch team are working on this, we’d still like the auth to come from within the ruby application via Warden or something so we’re not doubling up on auth logic. It’d also be awesome if we can mount couchdb in my applications url space so that we can use couch directly from the browser via ajax in my main application. Something that I’ve actually wanted to do for a while.

So I started hacking on implementing this in Pancake while Lincoln got over his jet lag. When he got up he showed me what he’d done on the plane as a lambda based rack application which sorted the issues I was having, and between the two of use we had a Pancake stack that was proxying to couchdb in pretty short order.

ProxyStack was born.

The first thing to do is get ProxyStack 0.2.0 or greater. It’s on gemcutter so

gem install proxy_stack

should do the job. This should bring in Pancake if you don’t already have it too.

Lets make a simple couchdb proxy from a config.ru file.


echo "require 'rubygems'; require 'proxy_stack'; run ProxyStack.stackup" > config.ru
unicorn -p 5000

Navigate on over to http://localhost:5000/ and http://localhost:5000/_utils to see your couch app proxied through your rubies :D

That’s kinda cool. We could also mount it at a different url by setting up a container stack and mounting it in there.

require 'rubygems'
require 'proxy_stack'

class ::ProxyContainer < Pancake::Stacks::Short
  router.mount(ProxyStack, "/my_couch")
end

run ProxyContainer.stackup

Restart unicorn (or just use shotgun) and your couchdb is now available at http://localhost:5000/my_couch/ (note the trailing /)

You’ll see that when you mount it away from root, the /_utils resource doesn’t really work. That’s because couch doesn’t know it’s mounted and generates the urls without the mount path appended. That doesn’t matter for what I want though, which is accessing the database itself via json calls.

Now, say we want to augement that so that we have an “/admin/users” resource on our stack. Maybe we want to provide a signup form or something similar, or maybe a login form. You can just add actions to the stack like you normally would. For example, lets inherit the ProxyStack into our own stack, and make some tweaks:


require 'rubygems'
require 'proxy_stack'

class ::MyProxy  < ProxyStack
  get "/admin/users", :name => :admin_users do
    "You're in #{stack_class} at #{url(:admin_users)}"
  end

end

class ::ProxyContainer < Pancake::Stacks::Short

  router.mount(MyProxy, "/my_couch")
end

run ProxyContainer.stackup

You’ll notice that I’m prefixing the class definitions with “::”. This is to put them into the global space. When loading a rackup, they’re loaded into an anonymous class/module. You can get around that by not declaring classes directly in your config.ru file.

Now if you mosey on over to http://localhost:5000/my_couch/admin/users you’ll see the results of the new augmented action on the stack. Neat.

The other thing we wanted to be able to do is authenticate / authorize calls to couchapp from within our proxy. Lets say we’re using Warden for the authentication. We can use a before_proxy hook to add an authenticate! call, allowing a host application to specify what that means, and just using it in our proxy stack.


class ::MyProxy  :admin_users do
    "You're in #{stack_class} at #{url(:admin_users)}"
  end
end 

Here, we’ve added a before hook to show authentication via warden, and also to print out a little message to the log. The hook is run in the action context so it’s like we’re right inside the action. We’re not going to setup warden inside this article so we’ll leave it commented out. You can add as many before_proxy hooks, and after_proxy hooks as you like.

So now we’ve seen it as a raw rack app, inherited into a container app, augmented and authenticated. But what about actually using it in X% of ruby apps… Rails. Well, because it’s rack, you can use it in rails as a metal. Go on over to your application and generate a metal:

script/generate metal proxy_metal

Make sure to require proxy_stack in your environment in config.gem and setup your metal to look like this:

class ProxyMetal
  class MyProxy < ProxyStack
    before_proxy do
      puts "BEFORE PROXY FOR #{stack_class} with #{request.path_info}"
    end
  end

  class CouchProxy    < MyProxy; end

  class ProxyContainer < Pancake::Stacks::Short
    router do |r|
      r.mount(CouchProxy,   "/couchdb")
    end
  end

  def self.call(env)
    @app ||= ProxyContainer.stackup
    @app.call(env)
  end
end

Fire it up and head over to http://localhost:3000/couchdb/ and you’ve got your proxy from within rails. It’s important that you don’t mount it at “/” in rails, otherwise every request will first check couchdb before being passed onto your rails app!

But, why did I put so much cruft into the Rails Metal one? You could have setup the rails metal to be this


class ProxyMetal
  class ProxyContainer < Pancake::Stacks::Short; end
  ProxyContainer.mount(ProxyStack, "/couchdb")
  
  def self.call(env)
    @app ||= ProxyContainer.stackup
    @app
  end
end

but I wanted to demonstrate something else. Proxy stack it turns out, can proxy whatever http request you like. This is for demonstration purposes only however. Before you do this, get permission or own the other site. I take no responsibility for your actions if you decide to try and hijack content, which incidentally, would make you a complete douche bag.


class ProxyMetal
  class MyProxy < ProxyStack
    before_proxy do
      puts "BEFORE PROXY FOR #{stack_class} with #{request.path_info}"
    end
  end

  class CouchProxy    < MyProxy; end
  class TwitterProxy  < MyProxy
    configuration.proxy_domain = "twitter.com"
    configuration.proxy_port      = 80
  end

  class ProxyContainer < Pancake::Stacks::Short
    router do |r|
     r.mount(TwitterProxy, "/twitter_proxy")
     r.mount(CouchProxy,   "/couchdb")
    end
  end

  def self.call(env)
    @app ||= ProxyContainer.stackup
    @app.call(env)
  end
end

Now you have http://twitter.com mounted in your application at http://localhost:3000/twitter_proxy/

Pancake’s Console

November 19, 2009

One of the really great things that Rails has done is provided us with script/console. This is such a handy utility and it works really well.

It’s no secret what I think of rack, but still, I’ll say it again. I think Rack is awesome, and although the spec is simple, when we conform to the spec we get new things from out of left field :D Enter (from field left) racksh This is pure gold IMHO. Install in the usual way:

$ gem install racksh

If we go back to the app we built up in my last post lets see how we can get a console for it.

∴ racksh
 ~ Loading Development Environment
Rack::Shell v0.9.4 started in development environment.
>> 

See the awesomeness of rack there? Racksh does a very nice job at providing a console to all rack applications who follow the spec and also provide a config.ru file. Also, since it’s an awesome tool, it makes your fully stacked up application available at $rack including goodness from Rack::Test so you get a whole bunch of methods for free :) Lets try a few out.

∴ racksh
 ~ Loading Development Environment
Rack::Shell v0.9.4 started in development environment.
>> WagyuBlog::BlogEntry.all
=> [#]
>> VegetableBlog::BlogEntry.all
=> [#]
>> response = $rack.get("/wagyu")
# snip response
>> response.status 
=> 200
>>

I’d suggest everyone checkout the docs and readme for both Racksh and Rack::Test to see what you can do with these great tools, and try them out. Use them with such frameworks as Sinatra, Rango, Pancake and many others :)

Inheriting and Mounting Stacks

November 18, 2009

One of the useful things about Pancake is that you can inherit your full application. This includes routes, bootloaders, configuration, paths and a few other things I won’t expound heavily on here.

What I want to go through is how to use it. Before we get started, make sure you have at least pancake 0.1.20

Since my initial post, Jack Dempsey has written a small test blog for Pancake. It’s not going to upset wordpress at this point (maybe one day), but it will serve to demonstrate inheriting and mounting gem’d stacks. The test blog stack is at http://github.com/jackdempsey/pancake-blog

Install Pancake-Blog

$ git clone git://github.com/jackdempsey/pancake-blog.git
$ cd pancake-blog
$ gem build blog.gemspec
$ gem install blog-x.x.x.gem

Once we’ve got the blog gem installed in the system, lets do the bare minimum to get it running in a container app. Move to another directory:

$ pancake-gen micro blog_container
$ cd blog_container

In pancake, you can progressively add directories of a larger app and they will just work. Lets setup a config.rb file and include an AR connection.

$ mkdir config
touch config/config.rb

Here’s an example of what you can put in there to get the connection setup.

require 'activerecord'
require 'blog'

ActiveRecord::Base.establish_connection(
  :adapter  => "mysql",
  :username => "root",
  :password => "",
  :host     => "localhost",
  :database => "pancake_blog_development",
  :encoding => "utf8"
)

You’ll notice here that we’ve also included the blog gem. Next lets mount the blog in the application. Lets get the blog_container.rb file looking like this:


class BlogContainer < Pancake::Stacks::Short
  add_root(__FILE__)
  initialize_stack

  router.mount(Blog, "/blog")
end

Initializing the stack will load the config file and mount applications etc. That’s almost it. Before we actually fire it up though, we need to create the database.

$ rake -T
$ rake blog:bootstrap

Pancake will load the rake files from the gem and make them available to you once we’ve mounted the Blog in the router. Now we can start the application :)

$ unicorn -p 5000 

Head over to http://localhost:5000/blog and you should see a shiny new blog. Enter a few posts and you can see it’s a very very simple app.

Inheriting Stacks

It’s kinda cool that we can mount an app and stuff. But inheriting full apps could be a bit cooler. Lets get that happening. Lets get the blog_container.rb file looking like this:

class BlogContainer < Pancake::Stacks::Short
  add_root(__FILE__)
  initialize_stack

  get "/" do
    render :root
  end
end

class WagyuBlog     < Blog; end
class VegetableBlog < Blog; end

BlogContainer.router do |r|
  r.mount(WagyuBlog,      "/wagyu")
  r.mount(VegetableBlog,  "/veggies")
end

We’ll also need to add this to the bottom of the config/config.rb file.

Blog.initialize_stack

This makes sure all the models in Blog are loaded before we inherit.

Ok, so lets start up the app again (with unicorn) and head over to http://localhost:5000/wagyu Aaaannnddd… The posts from before have disappeared… That’s actually intentional. It’s done that way in pancake-blog so you can mount multiple blogs in the one process without clashing the posts.
The model Blog::BlogEntry (the post) is inherited along with Blog so we end up with WagyuBlog::BlogEntry and VegetableBlog::BlogEntry. By doing this we’re segregating the data via STI.

Try it… Go to http://localhost:5000/wagyu and make some posts. Now scoot on over to http://localhost:5000/veggies and put in some posts there. Flip about a bit and you’ll see it’s actually separate.

That’s kinda cool, but we can do better. It’s not much good to have them all behaving the same all the time, and having all the stuff in the blog_container.rb file isn’t the most awesome. Lets mount them through the mounts directory. Create a “mounts” directory and a sub-directory for each blog:

$ mkdir -p mounts/wagyu-blog
$ mkdir -p mounts/vegetable-blog

Now, lets make the blog_container.rb file look like this by removing the inherit calls:

class BlogContainer < Pancake::Stacks::Short
  add_root(__FILE__)
  initialize_stack

  router do |r|
    r.mount(WagyuBlog,      "/wagyu")
    r.mount(VegetableBlog,  "/veggies")
  end

  get "/" do
    render :root
  end
end

Now, lets create a pancake_init.rb file in each of the sub-directories in “mounts” In them, we’ll put


# mounts/wagyu-blog/pancake_init.rb
class WagyuBlog < Blog
  add_root(__FILE__)
end

# mounts/vegetable-blog/pancake_init.rb
class VegetableBlog < Blog
  add_root(__FILE__)
end

Start the applciation again. You’ll see that the behavior is the same as doing it the other way. So, what exactly did that buy us? We’ve added a root to each of the blogs. That means we can start tweaking the views and add models / other files etc. We’ll just do one, but it will work for either, or both.

If you look at the pancake-blog source. You’ll see that the view inherits_from :base. Lets replace the base of the wagyu blog. Make a views directory in mounts/wagyu-blog/views and create a base.html.haml file. erb and erubis works just as well. Create your template, remember that you need to create a content block for at least :content (because of the child template in pancake-blog) That means you’ll need something like:


# snip template stuffs
- content_block :content do 
  %p
    No Posts Found

# snip

When you’re playing with templates, it’s a good idea to restart the app with shotgun so that the templates are re-compiled on each request.

shotgun -s thin -p 5000

I’ve made my mounts/waygu-blog/views/base.html.haml look like this:

!!!
%html
  %title A blog about Wagyu

  %body
    %h1 A Blog about Wagyu

    - content_block :content do
      %p
        No Posts Found

    %hr/

    .footer
      Here's My Footer

Not a very dramatic template I know, but you can see that it’s different when you render it. The VegetablesBlog at http://localhost:5000/veggies and see that it’s still the same as it was before we modified the templates for the WagyuBlog.

You can change any and all of the templates as you see fit for each inherited mount individually. You can even add a common root to both of them (or the parent class) and tweak there to have it affect both of them.

Getting Started with Pancake

November 15, 2009

It’s been a really busy week at work this week :) Now I’ve got a chance to write a quick “getting started” post I thought I’d show everyone how to get going with Pancake.

Pancake is designed to be flexible. You can have a single application contained in a config.ru, a two file config.ru + application file, or something with a bit more structure like a “micro” or “short” stacks.

I’ll just focus on the micro and short generated stacks today. They’re both Pancake “Short” stacks, but the generators are suited to different purposes. To get started you can use pancakes generator “pancake-gen”

Micro

$ pancake-gen micro my_stack

This generator will output enough code to generate your Short Stack app, make Passenger happy, and be able to mount it inside another stack.

The main application file here is “my_stack.rb” and there’s a Rakefile, config.ru, and public and temp directories for Passenger. You’ll also see a pancake_init.rb file. This file is not used unless you’re mounting this inside another stack (we’ll get to that later)

If you take a look in the my_stack.rb file, you’ll see that there’s a simple action in there. With any kind of Short Stack, just return your response at the end of the action and your gold. Pancake and Rack will look after the rest.

Views are stored in the “views” directory in the <template_name>.<format>.<render_engine> format.

Short

$ pancake-gen short my_stack

When you generate a stack with the “short” argument, you get a bit more structure. The biggest difference is that a “short” generated stack has everything you need to make the stack into an independent gem.

$ rake -T

Shows you all the Jeweler goodness for gemming up your stack.
The real working directory in this kind of stack is at lib/my_stack There’s a few more directories generated in there with this one. Mostly it’s just a bit more structure for a larger application.

Mounted Stacks

Ok so after that extraordinarily brief rundown of where each of the generated stack types fit, lets generate and mount these bad boys.


$ pancake-gen micro app_container
$ cd app_container
$ mkdir mounts
$ cd mounts
$ pancake-gen short mounted


Great, now the "mounted" stack will get loaded when you start up the app_container stack. The AppContainer stack will load all the mounts by loading mounts/*/pancake_init.rb You can use that to load anything you want. Another Rack application like Sinatra, a custom bit of kit. Even ActiveRecord plugins if you wanted.
As it stands right now, there's no way to access the mounted app in the browser though so you need to mount it in url space.
Open up the app_container.rb file and get it looking like this:


class AppContainer < Pancake::Stacks::Short
  add_root(__FILE__)
  initialize_stack

  router do |r|
    r.mount(Mounted, "/mount")
  end
 
  get "/", :_name => :home do
    "You're in the App!"
  end
end

Here can see we're also initializing the stack. When we do that, all the mounted apps and models are loaded which we need to do prior to mounting the apps in the router.

You may want to edit the "mounted" app to show something interesting.


  # mounts/mounted/lib/mounted/mounted.rb
class Mounted
  get "(/)" do
    "You're in Mounted!"
  end
end

I tend to use shotgun when developing. This is a code reloading rack adapter and suits very well :) I also tend to tell it to use thin and start on port 5000. Go back to the "app_container" directory, so you're in the directory with the config.ru file.

$ shotgun -s thin -p 5000

Now you can visit http://localhost:5000/ and http://localhost:5000/mount to see your nice shiny new mounted application :)
When you're ready to give it a non-reloaded squirt, I use unicorn to start.

$ unicorn -p 5000

it's much faster.

Although we haven't gone into the ins and outs of how to write pancake apps. At least we can now generate them and get started.

Pancake Stacks… A great tasting blog

November 9, 2009

Welcome to my blog about pancake :)  Tasty Tasty Pancake.  I’ll get into what Pancake is shortly, but I thought a little background might be in order.

Some of you may remember me from the Merb framework.  I was privileged to work on the Merb framework for a significant amount of time.  We had great growth and had some great ideas, implemented some really good stuff.  ahh.. great times…

Well since the announcement of the merger with Rails, I started looking at developing Rails.  Unfortunately the ideas and direction that Rails is going for Rails 3, whilst awesome, is not the direction that I wanted to go for my free time project.  The idea of fully mountable applications was a very strong lure.

I really wanted a way to mount applications, to have small, focused pieces of code that are loosely coupled.  I don’t just mean loosely coupled pieces of the framework, but also loosely coupled pieces of application code.

We really started to see this with Merb Slices and Rails engines.  Unfortunately, these just don’t take it far enough.  With these methods you cannot just pick up an application, and mount it inside another application.  They’re not good rack applications, and not really able to stand on their own two feet as applications in their own right.

There is something that does allow us to do that though.  Rack.  Rack allows us to create extraordinarily modular code.  The fact that Rails, Merb, Rameze, Sinatra, Rango and other frameworks are all built on Rack is telling.  Rack built in such a way that code re-use and mountable applications and sub applications are natural within Rack.

So, since shortly after the merger, once I realised that Rails was not the project for my spare time, I started working on a project that implements the ideas I had for creating mountable applications.  The applications should be self contained rack applications, able to function as gems, able to pick up an entire application and mount it inside another, able to inherit the whole application, take care of the low level plumbing, and lastly, let you create your own type of application when required.

Pancake


Follow

Get every new post delivered to your Inbox.