Thursday, 24 December 2009

New Gadget — Wireless IP Camera with Night Vision

Has finally treated myself with a new gadget, which is an extremely cool home CCTV with a night vision:

There're probably so many valid uses for it that I can't really come out with anything right now (apart of, obviously, keeping an eye on what's going at home while we're somewhere on holidays and receiving some notifications when something is moving in the flat while it obviously shouldn't), but hey, it's an incredibly cool gadget after all!

This specific model is a 3rd generation of Y-Cam's cameras and is finally 100% compatible with Macs, is able to transmit sound using industry-standard G.726. Furthermore, after a bit of hacking (on the client side) this cam's stream can even be viewed on the iPhone! (however, without the sound and in rubbish quality, but it's still better than nothing). 

Of course, I told Natalie that this is our New Year gift (as obviously it's a bit too late for Christmas) but she somehow didn't sound convinced. However, she didn't sound annoyed too — after all, it's a useful gadget, unlike the great many we already have in our household :-)

Anyway, here's a link to this particular model: http://www.y-cam.com/y-cam-knight-sd#Specification. This is a nice promo video — also worth viewing.  And finally, an all-excited review from one of my most trusted review websites (ironically, it's somehow found the way to it's name) — although it's a pre-previous model, it still gives some idea about what it is. 

A downside is — I will probably get the order only after New Year. Yeah, it's bad, I know, but it's better than nothing and this idea just happened to me this morning. So let's wait and see, and I promise a full detailed review with photographs and videos as soon as I get this cam into my hands.

Tuesday, 22 December 2009

Saturday, 19 December 2009

Reading (fan)fiction off-line in Stanza (and other ebook readers)


Some of us love fanfiction. Again, some of us don’t — so if you happen to be one of those, you probably won’t find this post of any interest to you. On the other hand, if you are interested — read on.


Today, will all these fancy ebook reading devices it’s really hard to keep yourself sticked to your monitor and reading from the websites such as fictionpress.com, fanfiction.net and so on. I felt that it is rather annoying too, so that’s why I’ve developed a fanficdownloader — a library, which helps to download fanfiction stories from multiple websites (list includes fictionpress.com, fanfiction.net, fanfictionauthors.net and others). Well, it was cool and all that stuff, but people didn’t like it (which is by no means surprising as end-users tend to like GUI applications rather than Python libraries with a clumsy CLI interface). So I went a step further…


… and developed fanfictionloader. Shortly, this is a web service, which does two things:



  1. Allows to download stories from all these different websites and convert them to industry-standard ePub or good old HTML using a very simple web interface

  2. Stores these stories for you (and you only) so you can always download them later.


So, if you want to read stories, say, in Stanza (which, surprisingly, supports ePub and supports it really well), your actions are pretty simple:



  1. Go to fanfictionloader and paste a URL of the first chapter of the story (only multi-chapter stories are supported at the moment). Before you get to the application, you must need to log in using your Google account (why you need it — you’ll understand later on).

  2. Click “Download” (you may need to provide your login and password for the website you’re downloading from).

  3. After link appears, don’t click on it but rather click on Previously downloaded fanfics link and make sure, that your story has appeared in the list. This list contains stories which were downloaded by you — and only by you.

  4. Go to your Stanza iPhone app, and add a new Source (like it is described in Stanza FAQ — for example, in bookworm section) with URL fanfictionloader.appspot.com

  5. Go to your newly added sources, login (again) with your Google account and click on the Previously downloaded fanfics. All your books are here. Download them.


Beauty of this approach is that you can download stories while you’re at your computer — and add them to Stanza later on, when you actually need them. Anyway, if you feel that something doesn’t work in the way it should — just ping me a mail/comment/tweet, I’ll be right here.


P.S. It's been my pet project for a long while and finally I decided to make it public. If you find it interesting and want to contribute to it, please let me know. Please also let me know if you terribly miss some functionality (say, you want to support some other website which isn't supported yet) — let me know too.

Wednesday, 18 November 2009

Python WTF?

>>> for i in range (0, 10):
...     hash(i)
... 
0
1
2
3
4
5
6
7
8
9
>>> hash(123324)
123324
>>> hash(785345345436845768)
785345345436845768
>>> 

WTF?

Tuesday, 17 November 2009

I Love Wikipedia

In mathematics, Stirling numbers of the second kind, together with Stirling numbers of the first kind, are one of the two types of Stirling numbers.

Isn't it a truly beautiful definition?

iPhone reinstall, contacts and SMS

I'll make a long story short today — I did a "new iPhone"-style firmware update, i.e. without restoring everything from the backup. iPhone started to work like a dream on 3.1.2. But I've lost few things which I didn't want to lose — in particular, my SMS messages. 

I found a backup of my iPhone which was done yesterday. Next I found a brilliant utility named iPhone Backup Extractor. I'll tell you what — normally I do not buy paid versions of software if the free version is doing what I can. This app doesn't have a paid version, so I just went and donated €5.00 to author — it's a brilliant piece of technology, dead simple and works. All what it does is just extracts  the data from the backup file(s) (there're zillions of them in iPhone Backup directory) in form they can be placed to the iPhone. Well - that's what I did. PhoneView allowed to copy and paste these files where they belong — and the next time I fired up my "Messages" application, I was pleasantly surprised by each and every SMS since I've purchased my iPhone.

Dear Mr Padraig Kennedy — you have saved my day. And I appreciate it. My best regards to you and to what you're doing.




Friday, 13 November 2009

XSS on new Perl.org — no input filtering??!

Apparently, guys from Perl should really learn how to filter the input (maybe they closed this hole right now, but at the moment of writing the link produces following picture):


Thursday, 12 November 2009

On men, women and feminism

Often when I am reading articles like this, I want to ask myself a question: why some people, who are (presumably) sane and educated, fail to understand one tiny little thing – men and women are indeed different. No matter how hard feminists will try, they will not change it, ever. I agree that sexism is bad. I agree that there should be equal opportunities for men and women. Furthermore, I strongly believe, that people are born equal, irrespective of their gender, skin colour, religion or habits. 

However, when I hear something "this can be offending to women!" I want to say only one thing: in most of the cases it is offending not to women, but to one particular woman (and, this woman was actively seeking for a way to get offended and shout about it). Why lots of women are happy to live their lives - and be happy in their way, rather than fight for some miraculous women rights? Megan Fox told one very good thing (in that article): "...women should be empowered by it, not degraded". And I think that's a brilliant true - we're born equal, yet different and it would be a very miserable world if there would be no difference between the man and the woman. 

(also, which is my personal opinion and I don't mean to offend anyone, but if woman acts like man, looks like man, speaks like man, and requires to be treated like man, then there's something clearly wrong with this woman).

Tuesday, 10 November 2009

Hadoop article

Recently I wrote a Hadoop article in Russian for one of very popular Russian IT blogs. After giving this idea a second thought, I translated this article (or, rather, first part of this article as the second is still in progress) to English and uploaded it to my website (Posterous format isn't very good for such long articles).

Check it here: http://romankirillov.info/hadoop.html

(and don't be mad for my clumsy English!)


 


P. S. in case you can read Russian: http://sigizmund.habrahabr.ru/blog/74792/

Monday, 9 November 2009

Building histograms using only MySQL


The query means a very simple thing: select count of matching rows, value of locations - grouped by locations field; rpad does all the magic - it basically says to add a certain number of '*' on the right of empty string, number of '*' is count(*). It's been divided by 15 to fit the bar into the console screen. That's it, chaps!

Friday, 6 November 2009

More Hadoop Mysteries - order of initialisation


Hey out there! Still not tired of my Hadoop experiments? Not yet? That’s another one for you!


What’d you think the difference is between two snippets of code? Say, this:


SomeCodeWhichChangesConfig.initialise(getConf()); 
Job job = new Job(conf, "MyHadoopJob");

// ... setting the job details

if (!job.waitForCompletion(true))
{
System.err.println("FAILED, cannot continue");
}


… and this:


Job job = new Job(conf, "MyHadoopJob"); 

// ... setting the job details

SomeCodeWhichChangesConfig.initialise(getConf());
if (!job.waitForCompletion(true))
{
System.err.println("FAILED, cannot continue");
}


No difference, you say? Not quite right, sir: the difference is that whatever you do to conf after creating a job will have no further effect. That is, Job constructor apparently copies all the data and doesn’t link your copy of Configuration object with it’s copy. Brilliant, no?


(and I spent a couple of hours trying to understand why distributed cache works properly in one app and doesn’t work at all in another). So you know now. Be warned.


Wednesday, 4 November 2009

XML input and Hadoop – custom InputFormat


Today I finally hit the task I was scared for so long — processing large XML files on Hadoop. I won’t tell you for how long I crawled the Internet trying to find some working solution… not that anyone wants to know? Eventually, I came out with the solution of my own — even though I hate re-inventing the wheel, in this particular case all the wheels I found were either square or were utterly incompatible with my model of car.


To make things more simple, I won’t include the full source code. I won’t even include the whole InputFormat class. So, to make yourself comfortable, please do following:



  1. Open LineRecordReader from org.apache.hadoop.mapreduce.lib.input so you can see it

  2. Open TextInputFormat from the same package.

  3. Create the input format and record reader of your own, just by copying and pasting the code from aforementioned classes.

  4. Change the constructor of your input format class so it’ll return your newly-defined record reader.


Now, we’re almost there. Now I’ll include the piece of code for nextKeyValue() which turned out to be the most critical method here. Hold on tight:


public boolean nextKeyValue() throws IOException
{
StringBuilder sb = new StringBuilder();
if (key == null)
{
key = new LongWritable();
}
key.set(pos);
if (value == null)
{
value = new Text();
}
int newSize = 0;

boolean xmlRecordStarted = false;
Text tmpLine = new Text();

while (pos < end)
{
newSize = in.readLine(tmpLine,
maxLineLength,
Math.max((int)
Math.min(Integer.MAX_VALUE,
end - pos),
maxLineLength));

if (newSize == 0)
{
break;
}

if (tmpLine.toString().contains("<document "))
{
xmlRecordStarted = true;
}

if (xmlRecordStarted)
{
sb.append(tmpLine.toString().replaceAll("n", " "));
}

if (tmpLine.toString().contains("</document>"))
{
xmlRecordStarted = false;
this.value.set(sb.toString());
break;
}

pos += newSize;

}

if (newSize == 0)
{
key = null;
value = null;
return fal
se
;
}
else
{
return true;
}
}


WTF — you will say? It’s the same code? Well — yes, and no. It’s almost the same. Take a look at this line:


if (tmpLine.toString().contains("<document")) 

and this line:


if (tmpLine.toString().contains("</document>")) 

This is where we actually split the document into chunks. Code is pretty-much self-explaining so I won’t add anything else.


Now, it’s not the most clean and streamlined solution and I probably will spend a while tomorrow making it more production-ready and good-looking, but compared to other solutions, it has few major benefits:



  1. It uses very little custom code (you remember, we copied and pasted all the classes?). Unfortunately you cannot just inherit the class — some fields are private, and we clearly want to modify them.

  2. It’s configurable — you can easily change the <document and </document> strings to anything else (and again, I will do it tomorrow, but now I feel too lazy).

  3. It works.


There’re few limitations of this approach. One of them is that if the document contains something like </document><document> it obviously won’t work. Another is — you still need to parse elements in your mapper (although you can easily change it by parsing records in your record reader into Writable-compatible class).



Have fun!


Update: As you can see, I have added a space in "<document " string constant – today I realised that "<documenttype" elements has been successfully used for splits, hence producing inconsistent results.

Huey colour calibration, pink colours and my Macs

I've got a thingie, you know... this colour calibration thingie named Huey. I got a while ago as a birthday gift from my friends, but when I switched to Mac two years ago it suddenly stopped to work (well it did but it produced that horrible pink cast – I couldn't use it and I didn't want to). Yet, a couple of weeks ago I decided that this not gonna work this way and sent a request to Pantone to do something about it. 


So, after some checks they sent me a replacement. I tried it at home and - yes, the same pink cast. Completely devastated, I decided to take it to work and try with my 24" HP monitor (connected to my MacBook).

I took it. And I tried. And it did work!!! Now my MacBook monitor and HP monitor look if not the same, but very similar at least. Actually, I did a very simple trick to make a non-pro version of Huey to work for two monitors: apparently, Pantone guys didn't know you can change the primary monitor (and it calibrates only primary monitor) just by dragging the menu bar in monitors' settings. I did it twice and it worked like a dream!

Seriously, colours are not perfect. I can see it even though probably none of my colleagues will see any difference after working with calibrated monitor for couple of hours. But – it's much better than original non-calibrated colours, and it makes me happy.

One day I'll buy a semi-pro Eye One Display, but until then I'll live with my Huey.

Thursday, 29 October 2009

Sequel Pro - fantastic Mac OS X front-end for MySQL

I bet you're either using MySQL, or going to use it or at least heard of it – I hardly believe anyone of reading my Twitter or blog are not exposed to this field at all. Hence, if you've ever used MySQL you have probably asked yourself a question – "where can I find a decent front-end application"? Of course, there's PhpMyAdmin as well as many others, and they're fine, really ... except they're web applications, and most of us still prefer something which runs on our machine locally - after all, it's totally different experience.

Now, if you're lucky enough to use Mac (and I see absolutely no reasons why you should not) – you've got an app for that, too! Sequel Pro gives you great database navigation, with ability to access server via SSH tunnel (as, obviously, not all servers are exposed to outer world and in fact, most are not). You can do practically all you want via it's UI - view and edit tables' structure, add or change indexes, view the content of even very large tables, and, of course, run SQL queries on the database. And, best of all - it's free! Try http://www.sequelpro.com and you won't regret – and I can just say "Thank You!" to the guys who made it available.

On the screenshots: Sequel Pro accessing MySQL database with loaded Wikipedia dump via SSH tunnel.





R.

Sunday, 25 October 2009

Buffalo Linkstation - USB2SATA adapter #fail

In feeble attempt to revive my Buffalo Linkstation home server, disassembled it to extract the disk and connect to my Mac – just to find out, that the USB2SATA adapter doesn't fit this disk (i.e. it is compatible from connector's point of view, but it's clearly too thick to get into tiny socket designated for SATA cable). Now waiting for SATA cable + power adapter from eBay – until the next weekend.




Autumn in Ravenscourt Park

Decided to have a short walk today – before diving in into studying. It was good :)







Wednesday, 21 October 2009

Sunset behind the office window

At the moment of writing all these unbelievable colours are already gone :) Too bad it didn't last for long.



R.

Tuesday, 20 October 2009

pv or pipe viewer — one of the most useful UNIX commands

Have you ever heard about UNIX pv command? I bet you didn’t — and I promise you, your life will change and you’ll become warm and fuzzy inside — because pv stands for “pipe viewer”.


Try to count, just how many times did you do something like:


grep -i something file.txt | gzip -c > output.gz

and wondering, just how the hell long will it take for it to finish? Well, for me it did happen quite often, especially when handling 10GB+ files (such as, say, Wikipedia dumps). That is exactly when pipe viewer comes to the scene. That’s how it looks:


[user@host wikipedia]$ pv enwiki.xml | wc -l
767MB 0:00:16 [67.9MB/s] [> ] 3% ETA 0:07:58

It’s the most simple use case — but so far one of the most useful ones (if you can say that about the use case). Say, you pipe more than one command and want to display the progress separately for every of them — no problems, you can do it! — as well as many other things.


Check out the pv’s webpage: http://www.ivarch.com/programs/pv.shtml — to me it’s one of the little utilities which make my life substantially easier.

Monday, 19 October 2009

The most unusual return form I've ever seen

I can't really comment on it - so I will just cite the text of the original email. I've never, ever, ever seen anything like that before.



This email has been sent to you by Griffin Technology to help facilitate testing and/or replacement of merchandise.

Generally we handle returns for our international customers a little differently than for our domestic customers. We assure you that this is our preferred method for handling your return as it will help save you money (shipping costs) and the hassle of having to send the product back to us. It also has the added benefit of helping us to ensure that your unit is truly non-functional.

First, destroy your defective product. Yes, really destroy it. A hammer works well for destroying most products however feel free to use whatever tool you are most comfortable with. This can be a great opportunity for you to express yourself creatively and take out some frustrations at the same time. Also, you may want to use eye protection and protect yourself from any flying debris that may result from this process.

Second, document the destruction. We recommend taking a digital picture of the destroyed unit while it is laying on a printed copy of this email so that the RMA number and other information are visible. Also, take a digital picture (or scan a copy) of your receipt.

Third, send us the pictures. Just reply to this email and attach the digital pictures of the destroyed product on the printed email and the receipt along with any other pictures that we might enjoy, such as you holding the destroyed product or the town where you live. Of course including the extra pictures is optional but feel free to include them if you want to.

Once we receive the email and verify the photos we will send a replacement to the address that you have provided us below. In order to ensure that we receive your email, please limit the size of the email to around 3MB. If we have not received a response from you within 30 days from the when this email was sent, your request for a replacement will automatically be removed from our system. So please respond as soon as possible.

Processing may take up to 2 weeks once the photos are received.

Thank you,

Griffin Technical Support
support@griffintechnology.com



Indeed, I've sent them the photographs of (thoroughly) destroyed cable and received confirmation, that the new one will be despatched soon. I don't know, how long will it last, but the approach to returns I've seen from Griffin - is truly admirable!

Probably, the most horrible UNIX command I've ever written

pv enwiki.xml | grep -C1 '<title>' | tr -d ' ' | sed 's/<title>//' | sed 's/</title>//' |  
sed 's/<id>//' | sed 's/</id>//' | sed 's/<page>//'  | tr -d ' ' | grep -v '^$' | 
tr 'n' 't' | tr '-' 'n' | grep -v '^$' > title_id.txt 

I know it's probably as efficient as spaghetti code is - but hey! It works and does what I want it to!


Saturday, 17 October 2009

Palm Pre in O2

Got my hands on highly acclaimed Palm Pre in O2 Store. So, it's not yet another iPhone killer (which mostly killed themselves), but - finally, it's a real competitor to Apple Phone.

The mobile phone looks pretty sturdy, is significantly smaller than iPhone though thicker too (see photographs of the phone in hand). Hardware keyboard is good and responsive, but to my taste buttons are too small - at least, iPhone keyboard looks preferable to me.

Phone feels very good in hand - probably, that's the case when size does really matter. Web OS seems to be fairly fast and responsive, although I didn't have much time to play with it properly. As I said, this phone finally appears to me as a first real competitor to iPhone - due to combination of size, weight, hardware keyboard and well-recognised brand (c'mon - Palm been at this market long and long before Apple has ever though about the iPhone!) I believe Palm Pre will find it's customers. All what I can say - it's really great, after all, Apple's been on it's own for far too long in this market!

(thanks to my lovely wife for helping me out with these photographs!)






Friday, 16 October 2009

Copying compressed file via SSH to Hadoop HDFS

Had probably re-invented the way of copying gzipped files via SSH to Hadoop's HDFS:



gzip -c file.txt | ssh hadoop.gateway.host 'gunzip -cf - | hdfs -put - input/file.txt'


And all the way back:


ssh hadoop.gateway.host 'hdfs -cat output/result.txt/* | gzip -c' | gunzip -c - > resulttxt

Friday, 9 October 2009

Thursday, 8 October 2009

i2reader - ebook for iPhone with dictionary

In my opinion, pictures are pretty much self-explaining. To be brutally honest, software isn't 100% ready for your average user -- it took me a while before I figured, how can I install a dictionary; also, widely-advertised on RipDev's website i2Librarian is ... well, let's say it's even less ready than i2reader itself.

On the other hand, once you've set everything up -- it just works as designed with no problems whatsoever. To lookup a translation just tap-and-hold the word; you can turn pages by sliding them (or tapping left and right sides of the screen, which can be customised in app's preferences) -- that's pretty much all you need to know about i2reader.

Ah yes, and English translation sucks :) I know, these guys are Russian, but after making a brilliant software surely they could spend a little time to find someone who can make a decent translation, couldn't they?









Monday, 5 October 2009

For switchers, who desperately miss their Start menu

It's simple, really - especially now in Snow Leopard. 

1. Fire up your Finder and go to Macintosh HD
2. Grab Applications folder
3. Put it into your dock next to the Trash bin, Downloads, etc.
4. Wait few seconds while it loads all the icons
5. Right-click it (or option-click it although if you're switcher, you shouldn't care about old-school Mac terminology) and select "View content as" -> List. 
6. Click the icon and see the menu.
7. Enjoy!

...
...

N-1. Though many find using QuickSilver easier and more simple to invoke your applications.
N. Personally, I use Spotlight as I hardly need QuickSilver's niceties, and good ol' Spotlight works better and faster.
N+1. Drag the app icon to your Dock, dammit!

P.S. did I ever say I am trying to promote my uni coursework for the phrase "mscais search optimization" ? :-)



Thursday, 1 October 2009