In a previous post I discussed the benefits of using membase server.  I mentioned that I was using the "memcached" bucket type instead of the membase bucket type, because I wanted to save money.  I was using EBS backed EC2 instances, and while the membase writes to disk do not impact performance significantly, disk I/O in EC2 costs you money if your using EBS.

Instance-store to the rescue.

Instead of using EBS, you can use the local "instance store" to back your root device.  AWS does not charge for I/O to instance-store. The downside, is the data is not persistent (if your instance dies/terminates, so does the data on the root device).  The good thing is membase supports replication - so if you do need to make sure your data does not go away, simply enable replication (its a SINGLE checkbox in membase :)

To summarize:

• Using instance-store instances to create your membase cluster will give you the ability to use membase buckets without the I/O cost
• Using membase buckets is good because 1) has more features 2) in my personal opinion, membase buckets are more stable because they are more common, and thus have much more burn-in out in the field
• Making your own instance-store backed AMI is a pain, but its doable.

There are lots of great resources out there for developing CakePHP apps in eclipse - but they are outdated and scattered.  I took the time tonight to setup my environment, so I figured I'd blog about it.  Below are the steps I took to get thinks as I like 'em. I run Ubuntu 10.10 64bit, but these directions should apply to any OS.  This requires you have CakePHP 1.3.X extracted someplace on your box.

1. Install Eclipse 3.7 (indigo). If your just gonna do PHP development, get the classic version.
2. Help > install new software > work with: all available sites > search for PDT. Install PDT.
3. Optional: Install Subversive if you use SVN, or install Egit if you use git/github. Subversive is available in default indigo eclipse repo. For Egit, i'd recomend using the http://download.eclipse.org/egit/updates repo.
4. Setup your PATH to use the cake console (optional, requires php5-cli to be installed).  This allows you to call 'cake bake help' for example, from your terminal.  Edit ~/.profile and add to the bottom of the file

   PATH="/path/to/cake/runtime/13/cake/console:$PATH"

5. Setup eclipse so it can run 'cake bake directly' from the IDE.
   1. Open eclipse, go to your PHP explorer perspective.  From the menu choose Run > External Tools > External Tools Configurations
   2. Click Program, then new program (white page upper left). Give it a name like Cake bake.
   3. In the Location field put the full path to the cake console script.  I have 1 shared runtime on my box for all my CakePHP 1.3 based apps. Ex: /opt/cake/runtime/13/cake/console/cake. Note: If you typically run diff versions of cake or have the cake runtime directly in your project, you can insert variables in the location box (like workspace).
   4. Under Working Directory put ${project_loc}/app
   5. Under Arguments put 'bake'
   6. Now click Apply then close.
   7. Test it out by clicking on a cake project (must have the structure of 'project name' with a subfolder called 'app'), then clicking Run>External tools>Cake bake.  This will open a console that you can type in.
6. Repeat Step 5 for other cake console things you would like to use from the IDE (like 'cake console' for example)
7. Setup syntax highlighting for .ctp files:
   1. Window > Preferences > General > Appearance > content Types > Text > PHP Content type > Add.. , then put in *.ctp.
8. Install Open Cake File eclipse plugin to enable fast switching between your MVC classes/files. This is not well known, but its a nifty plugin.

Thats it! If you know of any other cool things email me, and I'll add em. Note: I left off code completion intentionally, as I big fan of the implementation of the solutions I've seen.

I've used memcached quite frequently in projects, it is a great product.  I heard of a memcached compatable product called Membase about 8 months or so ago, but did not have time to try it out until last week.  All I can say is WOW. I'll never use stand alone memcached server(s) again.

Highlights

• crazy easy to install and make a cluster.
• 0 changes to your app code. Operates seamlessly with memcached protocol.  If you want to take advantage of advanced features, you need to modify app code.
• you can dynamically add and remove nodes without losing all your keys/data.
• 2 bucket types:
  1. Membase: supports data persistence (writes them ionicely to disk) and replication (one node dies, you dont lose your key/value pairs). It sends data to disk as fast as it can (while giving priority to getting data back from disk). This is done asynchronously (with an option for synchronous), so clients shouldn't be able to perceive a difference between Membase and memcached data buckets.
  2. Memcached: no persistence or replication. all in memory. I would highly recomend going membase bucket unless you have some I/O concerns (like you get charged for I/O in the cloud).
• Awesome admin web UI. You can get feel for it in this screencast (this is for v2.0, but its similar). It updes real time, gives great stats, and is intuitive.
• lots of documentation
• helpful community (I found the freenode #couchbase irc channel to be really helpful) 

How I'm using Membase

I have a membase cluster setup in EC2. All m1.smalls. I would not recomend smalls if you can afford going larger, as smalls only have 1 cpu core and have "moderate" I/O.  Because of the I/O cost ($) in EC2, and because I don't need persistence or replication for my apps (at this point), I went with membase bucket.  I'd highly recomend going Membase bucket if you can.  I've spoken with one of the Couchbase dev's, and in practice they have not seen much performance difference between the membase and memcached buckets.

I've had pretty good transactions/sec with just 2 m1.smalls - easily over 1,250/second.  I'm sure I could get more no problem. I get roughly the same numbers when using membase and memcached buckets.

I have 2 autoscale groups setup to keep my cluster highly available AND easily scale up/down based on load.  Essentially, I have one "master" node that I assign an elastic IP. Its in a auto scale group of min 1/max 1 - so if there are problems it gets replaced and automatically gets the EIP. I then have a second autoscale group that on startup joins the cluster via the master node EIP. This is a super cheap way of getting enterprise class HA and scale - 4 data centers (availability zones), "ip failover".

I could have the cluster automatically scale up based on load, but I decided not to go this route based on the fact that when the cluster topology changes it needs to be "rebalanced".  This can be a resource intensive task - so I would not want it happening while I was under load (the reason it scaled up in the first place).  For now I just monitor load manually, and scaling up is 1 command to add a node, and a push of a button to rebalance (when I deem its a good time).

If you do decide to run membase in the cloud, you must read Membase in the cloud docs.

How my app connects to Membase

My apps that consume the membase cluster are php based. Spefically I use php-fpm and nginx.  FPM has some great features like fastcgi_finish_request(), but thats another blog post in its self.  Anyways, Membase has a great app called Moxi. It runs either client side (where your webapp is) or server side (on Membase cluster nodes).  If you want to get up and running fast (without touching your existing app), just point your app at any server in the membase cluster (port 11211 by default) and it will use the moxi on the membase server without you even knowing.

If you want to improve performance, and take advantage of connection pooling - check out client side moxi.  Client side moxi will stay in contact with the cluster, so if any node goes down it will handle it transparently for you. Also, because it stays in contact with the cluster, when interacting with a  key, it goes directly to the correct node that has that key (performance++).  Client side moxi also has connection pooling - which is great for fast CGI processes.  After installing client side moxi, just have your app make a persistent memcache connection to localhost:11211 and your done.  Moxi acts as a proxy and does all the work.

In conclusion ...

• If you use memcache servers, using Membase in their place is a no-brainer.  Community version is free (if you can adhere to the open source licence). You can get everything up and running in < 5 minutes on Debain, RPM or windows based machines.
• When creating buckets, you should use membase type buckets.  Using memcached buckets does not make sense in most cases.
• Spend the time to research client side moxi. Also, you may be using a language that supports a type 2 membase client. If so you may want to use it instead of moxi.

I'm still not a membase expert, so if you have any thoughts/improvements please leave me a comment.

I often use JMeter to test web apps and web services - it is a great tool - easy to use (GUI based), java based (so runs on any platform), lots of function etc....

Now that most production level apps/services sit behind a load balancer (AWS ELB for example) JMeter will not really test all the servers behind the ELB without some configuration.  Why? Java DNS cache.  For load balancers that use a DNS based distrubution method (like round robin), your HTTP client makes a connection to the load balancer, and the load balancer says "hey use IP x.x.x.x".  Your HTTP cleint then caches that ip for future requests.

This is not good when trying to test your load balanced app, as you will only be testing one of the nodes that is behind the load balancer.  To get around this in JMeter, just change the JVM dns ttl by passing a properties file to JMeter. Example:

$ cat test.properties 

networkaddress.cache.ttl=0
sun.net.inetaddr.ttl=0

Now run JMeter via:
jmeter -q test.properties -t test_plan.jmx 

Thats it!

For your production apps only

Here are the steps to setup a linux headless display server to take screenshots of websites using http://cutycapt.sourceforge.net/ and Xvfb. This example is using ubuntu 10.10 64bit.

*this assumes you have cutycapt installed and working - as well as Xvfb

1. If your taking multiple screenshots (like as a service for example) you'll want to run Xvfb as a service.  Create the following file in /etc/init.d/screencap

2. Make it so it runs at startup
   
   sudo update-rc.d screencap defaults
   sudo chmod 700 /etc/init.d/screencap
3. Now you can restart, or run:

   sudo /etc/init.d/screencap start
    

4. Now from another terminal take the screenshot:

   DISPLAY=:1 /usr/bin/CutyCapt --url="http://rynop.com" --out="rynop.jpg" --out-format=jpeg --plugins=off --delay=4000

So why not use xvfb-run? I intend to take multiple screenshots, so starting Xvfb before every screenshot, then killing (which xvfb-run does) it alot of wasted overhead.  I see the Xvfb process only takes up ~ 7megs of memory - so its not really all that bloated.

Taking screenshots is pretty slow, as you have to wait for page to render.  If anyone has tips on how to improve performance please let me know.

Extra credit - monitor the service with monit

1. Create the following file: /etc/monit/conf.d/screencap.monitrc

2. Run /etc/init.d/monit restart

So I finally got Eclispe setup to debug my PHP5 fpm through nginx and I figured I'd share how to do it, since I was unable to find anywhere on the web that had clear step by step instructions.  These assume you have eclipse installed, as well as nginx with php5-fpm as well as the xdebug php modlue.  For me, php5-fpm was running under localhost on port 9000 on ubuntu 10.10 64bit.

1. Setup xdebug.ini
   
2. Restart php5-fpm
3. From eclipse:
   1. window > preferences > PHP > debug:
      Debugger: xdebug. Click configure and put port 9001 for XDebug port. And for Accept remote session (JIT) choose any.
      Uncheck "Break at first line"
   2. Next to server click PHP Servers. Click New, put in any name, and the URL you would put into your web browser to get to the page on your local system (mine for example is https://localhost.mydomain.com - I put an entry into /etc/hosts that maps localhost.mydomain.com to 127.0.0.1)
   3. window > preferences > General > We Browser:
      Clcik Use external, add your fav browser (I did new, /usr/bin/firefox).
   4.  From PHP perspective in eclipse, do run > debug configurations..
      1. Click PHP Web page and press "new launch configuration"
      2. Give it a name, choose:
         Server Debugger: XDebug
         PHP Server: <name of server u made in step 3.2 above>
         File: <the file in your workspace u want to debug - click browse button for help finding it>
         Uncheck break at first line
         Uncheck autogenerate, and put in a valid URL 

OK so now you should be able to press "debug" and your external browser will open (firefox in my case).  Use your browser and navigate to the php page where you have set a breakpoint. It should instantly flip to debug perspective in eclipse and hit ur breakpoint.  

Happy debugging. If you have any tricks that I don't know about please get ahold of me and I'll update this post.

Update 5/27/11: You gotta check out the easy x-debug plugin for Firefox. It allows you to enable remote debugging and/or profiling with 1 click. If you use this plugin, you can skip step 4 above. Just click the start icon in firefox, then request your page. If you have a breakpoint setup in your php project in eclipse it should automatically launch the debug view.

You know how as a developer each time you create a new web app you spend the first few hours downloading, installing, and configuring the important add-ons that you need for nearly any production-level project? When all you really want to do is get started on your kick-ass code! That's why a friend and I decided to do something about it. We're launching CakePlate, a open source distribution of CakePHP that's ready to rock, right out of the box.

Check out the announcement and learn more by visiting http://cakeplate.posterous.com/

Note: updated 9/9/11 for eclipse 3.7 (Indigo)

There are lots of great resources out there for developing CakePHP apps in eclipse - but they are outdated and scattered.  I took the time tonight to setup my environment, so I figured I'd blog about it.  Below are the steps I took to get thinks as I like 'em. I run Ubuntu 10.10 64bit, but these directions should apply to any OS.  This requires you have CakePHP 1.3.X extracted someplace on your box.

1. Install Eclipse 3.7 (indigo). If your just gonna do PHP development, get the classic version.
2. Help > install new software > work with: all available sites > search for PDT. Install PDT.
3. Optional: Install Subversive if you use SVN, or install Egit if you use git/github. Subversive is available in default indigo eclipse repo. For Egit, i'd recomend using the http://download.eclipse.org/egit/updates repo.
4. Setup your PATH to use the cake console (optional, requires php5-cli to be installed).  This allows you to call 'cake bake help' for example, from your terminal.  Edit ~/.profile and add to the bottom of the file

   PATH="/path/to/cake/runtime/13/cake/console:$PATH"

5. Setup eclipse so it can run 'cake bake directly' from the IDE.
   1. Open eclipse, go to your PHP explorer perspective.  From the menu choose Run > External Tools > External Tools Configurations
   2. Click Program, then new program (white page upper left). Give it a name like Cake bake.
   3. In the Location field put the full path to the cake console script.  I have 1 shared runtime on my box for all my CakePHP 1.3 based apps. Ex: /opt/cake/runtime/13/cake/console/cake. Note: If you typically run diff versions of cake or have the cake runtime directly in your project, you can insert variables in the location box (like workspace).
   4. Under Working Directory put ${project_loc}/app
   5. Under Arguments put 'bake'
   6. Now click Apply then close.
   7. Test it out by clicking on a cake project (must have the structure of 'project name' with a subfolder called 'app'), then clicking Run>External tools>Cake bake.  This will open a console that you can type in.
6. Repeat Step 5 for other cake console things you would like to use from the IDE (like 'cake console' for example)
7. Setup syntax highlighting for .ctp files:
   1. Window > Preferences > General > Appearance > content Types > Text > PHP Content type > Add.. , then put in *.ctp.
8. Install Open Cake File eclipse plugin to enable fast switching between your MVC classes/files. This is not well known, but its a nifty plugin.

Thats it! If you know of any other cool things email me, and I'll add em. Note: I left off code completion intentionally, as I big fan of the implementation of the solutions I've seen.

Few slick features that have been added (some were in 1.x):

• Automated playlist generator
• Bookmarkable Filters ex: added:<1week
• Randomize current playlist