Internet WorkShop

HDFS, high density file system, is useful for big data. However, hadoop fs is not quite there as a shell replacement. Today I kept getting the message

cp: When copying multiple files, destination should be a directory.

when trying to copy multiple files to a directory using

hadoop fs -cp /path/to/files/*  /path/to/destination/directory

Finally figured out that the problem was I had two spaces between the file list and the directory path, which made hadoop not see the directory path in the command. Aaahh.

gems/gdbm-1.2/lib/gdbm.rb:256:in `initialize': Empty database (GDBMError)

error occurs when trying to use

g = GDBM.new('somefile')

on an nfs-mounted partition. GDBM works fine on normal drives, just don’t try it on nfs-mounts. Posting this as I found nothing when I googled the error message, and wasted several minutes before I realized the problem. The error message may be specific to the ruby ‘gdbm’ gem, but the rule is a general one.

Note: I’m not a wordpress expert, just returning to it after several years without having touched PHP – and looking for the best way to quickly understand the flow of a wordpress site using buddypress and a few other plugins. Raw notes here, will be annotated as I progress…

http://fuelyourcoding.com/simple-debugging-with-wordpress/

Spell checking and typeahead are two of my top gripes with modern software. URL or bookmark completion is ok, that is when I’m in the ‘trying to remember’ mode. But when I’m in the flow of writing, having the computer guess what I’m trying to say is incredibly distracting and annoying.

Originally, I thought gmail was running auto-spellcheck for me, but it was the browser, in this case Google Chrome.

In Chrome, you turn off spellcheck under chrome://settings/language ; uncheck the ‘Enable spell checking’ box underneath the list of languages.

(You can also get to this screen advanced settings screen by clicking on the wrench in the upper-right, select ‘Preferences’ and ‘Under the Hood’, then click ‘Languages and Spell-checker Settings’ )

Update 7/14/11 – I believe this is fixed in rturk 2.4 – thanks Mark!
http://rubygems.org/gems/rturk
***
rturk, the Ruby gem for making calls to the Amazon Mechanical Turk API, uses a REST transport layer. That’s fine, but all calls are currently performed by a GET, which has a length limitation. When making calls that include long strings of data – such as the XML for a QuestionForm structure in a qualification tests – errors may occur with the non-explanatory message ‘400 Request Error’.

Was able to patch it by making a change to lib/rturk/requester.rb :

46,47c46,50
< RTurk.logger.debug "Sending request:\n\t #{credentials.host}?#{querystring}"
< RestClient.get("#{credentials.host}?#{querystring}")
---
> # RTurk.logger.debug “Sending request:\n\t #{credentials.host}?#{querystring}”
> # RestClient.get(”#{credentials.host}?#{querystring}”)
>
> RTurk.logger.debug “Posting request to #{credentials.host}:\n\t #{params.inspect}”
> RestClient.post(credentials.host.to_s, post_params)

A more robust fix might be to use POST only for longer requests, or make it an explicit option on the RTurk object

I’m not sure where the bug is, but when saving some binary data that was generated in a Rails3 before_create callback, it kept getting truncated in the actual INSERT INTO statement (though it appeared fine even in after_save callback, it was truncated in the database). Using PostgreSQL 8.4.7 with pg (0.9.0) and activerecord 3.0.5

Seemed like it could be related to this fixed bug, but my problem is on the save itself:
https://rails.lighthouseapp.com/projects/8994/tickets/611-cannot-write-certain-binary-data-to-postgresql-bytea-columns-in-2-1-0

In any case, found a simple workaround: uuencode the data first, and uudecode on loading.

   before_create do
      ... stuff that builds my_hash ...
      self.my_hash = Base64.encode64(Marshal.dump(my_hash))
   end

Then later, to reconstitute the hash,

   loaded_hash = Marshal.load(Base64.decode64(@record.my_hash))

In the helper file, ie tablenames_helper.rb, override the render_action_link method

 def render_action_link(link, url_options, record = nil, html_options = {})
    if link.parameters[:association] == :my_association
       # special handling here
    else
       super  # in case you have other associations for which you do want the default behavior
    end
 end

This confounded me for a while as it was easy to create a helper to override the link to associated records, but if the records did not yet exist the ‘Create New’ appeared hardcoded. After inspection of ./bundler/gems/active_scaffold-45451d963672/lib/active_scaffold/helpers/list_column_helpers.rb and ./bundler/gems/active_scaffold-45451d963672/lib/active_scaffold/helpers/view_helpers.rb the above override became apparent. Is there an easier way to determine this sort of thing?

After a bit of refactoring of one of my controllers in a Rails3 app, the layout mysteriously disappeared, with

Rendered vendor/plugins/active_scaffold/frontends/default/views/list.html.erb (188.5ms)

replacing

Rendered vendor/plugins/active_scaffold/frontends/default/views/list.html.erb within layouts/application (192.1ms)

in the Rails debug output. Turned out to be a simple error – I added a routine to initialize some instance variables and forgot the ’super’ at the end to call the parent class initializer:

def initialize
   ... my instance vars ...
   super
end

Just had to write a script to do some processing on a tab-delimited file that contained some Hebrew UTF-8 chars, including a json chunk containing some Hebrew. Major headache with Perl – either got ‘wide char in subroutine’ errors, or escaped codes in the JSON. Needed to compare the UTF-8 tab-delimited fields with specific pieces of JSON, and have them both handled as UTF-8. Tried use utf8; binmode(STDIN, ':utf8') and various other permutations, but couldn’t get the output to look the way I wanted while keeping all the subroutines happy.

Implemented the same thing in Ruby, and it just worked. I’ve been a Perl girl for years, but am leaning Ruby…

The instructions at http://www.cyberciti.biz/faq/linux-unix-set-java_home-path-variable/ are slightly out of date or incorrect. Instead of setting

export JAVA_HOME=/usr/lib/jvm/java-6-sun-1.6.0.22/bin/java # do NOT do this

as you might expect from their instructions, you should

export JAVA_HOME=/usr/lib/jvm/java-6-sun-1.6.0.22/