Rendering problem with Kdenlive

I really enjoy using the Kdenlive video editor. I think it’s a simple, straight forward, and yet powerful tool.

But just during the last few days, I had troubles for the first time and could not find a solution. When rendering my project, it kept failing with max_analyze_duration 5000000 reached at 5000000. I searched online, the error might be related to the size of the audio (>1h) Still, the project rendered at first, but after the final cut – with lots and lots of cuts – I started getting this error, thus, out of a sudden, I was not able to get my final video file out anymore. Was I threatened to having to do the entire work again or what??

Finally, today I found a solution! Better, a workaround.

I split up the file by adding guides – in my case three, but the number might depend on the project size – and then used the render feature “Guide zone” (at the bottom of the render dialog). So, I rendered three parts and used melt to put them together like this:

melt input*.mp4 -consumer avformat:result.mp4 acodec=aac ab=160k vcodec=libx264 vb=3000k

I’m using x.264 and AAC here – but you can also drop all those parameters and let melt figure everything out – it’s pretty capable, give it a try.

melt input*.mp4 -consumer avformat:result.mp4

If you have ffmpeg on your system, you can merge the files using the copy mode, so no quality gets lost – see details here. But even when using melt and thus re-encoding the video, you will not notice that it has been rendered in three parts and “melted” later. So I’m happy! Still a bit surprising that I could not find a solution online. So hey, hope this approach works for you too! 🙂

Excellent photos free to be reused

Looking for great photos for your next project or presentation? And you enjoy to be on the legal side of life? Check out this great websites with free stock photography!

Excellent quality: Pexels, Life of Pix, Stocksnap.io, Unsplash,
Gratisography (by Ryan McGuire).

Good quality: Cupcake.

Good, but…
A bit complicated to browse: Negative Space, Great shots food, but the licensing is not exactly clear: Foodies Feed.

And it’s all free to use via the Creative Commons Zero license (aka CC0). Even for commercial projects… and if you need even without attribution. Great stuff – truely free. 🙂

Sources: Slideshare blog and others.

Recover deleted photos

Shit happens… but for Linux users, there is a power tool straight from the US Air Force called "foremost" to help you recover (aka undelete) your photos easily!
Note: you’ll need sudo rights to install this software tool.

But most importantly: do not write any new data to your device (SD card, flash drive or whatever)! Otherwise, you’ll overwrite what is still there and lose your photos forever.

The steps:
Remove or unplug the media/drive/card from your comupter.
Open a terminal window.

Install the recovery software.

sudo apt-get install foremost

Create a folder to put the recovered photos in. Important: don’t put this forler on the drive where you want to recover photos from! See above…

mkdir recovered-photos && cd recovered-photos

Find the device: copy the following command to your terminal and press the [Tab] key twice

foremost -v -t jpg /dev/sd

It will say something like this

sda   sda1  sda2  sda3  sda4  sda5  sda6  sdb

Now, plug in your drive/card/medium and press [Tab] again, twice.
It should change to something like this

sda   sda1  sda2  sda3  sda4  sda5  sda6  sdb  sdc  sdc1

In my case sdc and scd1 showed up. Thus, sdc1 is the partition on the device to recover from. Thus, I my case the full command would be (You need to replace sdc1 with whatever showed up previously on your own terminal!)

foremost -v -t jpg /dev/sdc1

… press return and see the magic happen! Your photos will show up in a subfolder of recovered-photos. 🙂

Setting up a linux server to forward all incoming mails

Typical thing when having a new site up: you want to get whatever@yourdomain.com to end up in you normal inbox, say Gmail.

Here are the steps!
Note: If you are logged in as root, you do not need to use “sudo” in front of each command.

Preparations

It’s recommended to have a user account to receive the mail other than root.
So either choose a user you have created already or create a new one:

sudo adduser [username]

Does your provider require you to use an SMTP relay? (check the documentation, or ask). You need to know this to continue.

Install postfix

Postfix is a software package, a MTA, it handles incoming mail and forwarding.

sudo apt-get install postfix

And answer all configuration questions as below:
Server configuration type: If you use an SMTP relay, choose “Internet with smarthost”, otherwise “Internet”
System mail name: [yourdomain.com] (no subdomain)
SMTP relay host: [smtp.yourprivider.com] (only showing if you chose “Internet with smarthost”)
Root and postmaster mail recipient: The created/chosen before (not root)
Other destinations to accept mail for: keep the suggested defaults and add in front [yourdomain.com] and a space
Force synchronous updates on mail queue: no
Local networks: keep suggested
Use procmail for local delivery?: yes
Mailbox size limit: 0
Local address extension character: +
Internet protocols to use: all

Manual configuration

All edits with sudo if you not logged in as root.
Edit /etc/postfix/main.cf add following two lines to the end

virtual_alias_domains = [yourdomain.com]
virtual_alias_maps = hash:/etc/postfix/virtual

Create/edit /etc/postfix/virtual and add

@[yourdomain.com]    [your.email.address@somewhere.com]

Finally, after editing this file call

sudo postmap /etc/postfix/virtual

so that postmap actually applies your changes.

Notes

If you forward to Gmail, do not use the same address to send test mails that the mails will be forwarded to – Gmail will silently ignore/delete mail where the sender and recipient are the same and you will think the forwarding does not work.

Copying MongoDB from meteor.com

I started deploying my latest Meteor app umeedoo.net to my own VPS and wanted to use the data from my beta system hosted at umeedoo.meteor.com.
First of all, I was surprised how simple and easy it was to deploy using Meteor Up (MUP). Some weeks before, I spend an entire day on installing the Meteor environment on a Debian machine – now it’s less than 15 minutes! One caveat though, it did not work for me on Debian, I had to drop my previous Debian VM and replaced it with Ubuntu as MUP depends on Upstart and it seems rather tricky to replace Debian’s SysVinit with Upstart. Ok, different story, but a biiig thanks to Arunoda for the great work on MUP!!

So, how to db.copyDatabase(); the data from the original DB to yours?

The method signature is db.copyDatabase("[app-name]_meteor_com", "[your-db-name]", "[meteor-mongo-domain]", "[username]", "[password]");
I’ll assume “app” to be the app name for simplicity.
General approach: get the MongoDB URL, extract all the parameters above, log into your own MongoDB installation and use db.CopyDatabase(); to move the data over.

All commands need to be executed on the same machine. (FYI, the username and password get automatically generated and it did not work to get the URL on my machine and use the username and password on the server… strange though)
Thus, install Meteor on your server curl https://install.meteor.com/ | sh.
Then, get the MongoDB URL meteor mongo --url app.meteor.com
It will look like this mongodb://client-fd532e7d:683a9a4e-1604-5729-07fd-df1ba7dcbbf9@production-db-a2.meteor.io:27017/app_meteor_com
The parts are

  • username: client-fd532e7d
  • password: 683a9a4e-1604-5729-07fd-df1ba7dcbbf9
  • meteor-mongo-domain: production-db-a2.meteor.io:27017

Start a MongoDB CLI by typing mongo.
In my case, the database created by the app and MUP was called “admin” (i.e. [your-db-name]=”admin”). How to find yours? Use “show dbs” in the MongoDB CLI to see which one is yours.
Then write “use [your-db-name]” to switch to your DB.
And finally, using the data from above: db.copyDatabase("app_meteor_com", "admin", "production-db-a2.meteor.io:27017", "client-2fbeb6ef", "1bd44ece-b3fa-7d1b-0046-83b917927f35");
will copy the data over in a few seconds.
When I tried first, I got an error about a duplicate key (the deployed app had created some data already), dropping the entire DB fixed it. Also, the username and password are valid for a short time only, so maybe having to terminals open is a good idea.

References: Cloning MongoDB, Meteor dump, Clone DB from meteor.com, Copy DB from one server to another

Match.Optional for null

Securing my Meteor app, I came across a strange quirk of Match.Optional of the "check" package.
Example (in CoffeeScript)

Meteor.methods
  test: (x) -> check x, Match.Optional String

I expected the match to be successful when calling the method like this:

Meteor.call 'test', undefined

But instead, I got a "Match failed" exception. After debugging for a bit, it turned out that the undefined becomes a null when it arrives on the server.
Which is odd, of course—but also annoying as the check x, Match.Optional String fails for null (only undefined or String are acceptable).
To overcome this, here is a new Match.Optional that does the trick:

Match.OptionalOrNull = (pattern) -> Match.OneOf undefined, null, pattern

Also, I keep wondering: what’s the purpose of making Match.Optional fail on null in the first place?

Mac flaws – hardware and software

I started this list about four years ago when I first got in touch with OSX and the hype around it. The usual claim: it’s great usability. But it’s not that difficult to spot serious usability issues. There are not that many – so it’s worth making a list (as opposed to other systems ;)), but it should show clearly: OSX has a great user experience (aka UX), which is the actual and understandable reason for the hype — but that must not be mixed up with usability.

Missing features

In general, OSX can not be controlled with the keyboard only, thus you are forced to switch between keyboard and mouse/touchpad regularly.

Using keyboard to control OS X

  • No delete key on the Apple laptop keyboard (only backspace and Fn+backspace to simulate the delete key)
  • No key to show the context menu on a currently selected item; but some functionalities are part of the context menu only. My new computer (Lenovo) does not have a context menu key either–following the example of Apple??
  • Alert boxes that are not part of a running application can not be focused using Cmd+Tab or Cmd+<>, thus, it’s not possible to send them away with the keyboard once they lost the focus.
  • When using finder in tree mode and inserting a copied file, it will always be put in the root folder instead of the folder the user has highlighted; only the context menu’s entry “paste item” does that as expected but is not accessible via keyboard.
  • When deleting an item from a list using Cmd+Backspace the cursor gets reset to the topmost position in the list instead of moving to the previous or next item, thus making it very frustrating to clean a large, distributed set of items from a long list

Thus, working with keyboard only (which is the most efficient way while coding) is almost impossible. 🙁

Point and Click Interface

  • No context menu (right click menu) in menus like “Downloads” in the launcher bar, therefore it is not possible to open a docx file in OpenOffice instead of Microsoft Office)
  • Only since OSX Lion files can be dragged and dropped from the “spot light” quick start menu. Before, e.g. to use the spot light to search a document to be send via Skype it was necessary to click “Show all” first -> Finder window pops up -> drag item from there + close window afterwards.
  • When opening a folder in a finder window, there is no button to step up the folder hierarchy since OSX Lion.
  • Multiple selections aren’t supported in menus such as Downloads. Imagine you downloaded several files, now you want to move them to a particular folder. So you either drag and drop each separately or you open up an additional finder window, navigate to "Download" sort the results by date and then drag and drop the latest files.
  • When opening an image in the preview, it is not possible to forward to the next image in the same folder. Instead, the user is required to select all images in the finder window and then open them in the preview to be able to skim trough them. But multiple selection is not supported everywhere (see above).

Hardware

Despite that my Mac Book is called "Pro" it’s lacking basic pro features.

  • There is no docking station available
  • There is no way to have a back-up battery as the battery is soldered in
  • I always need to carry the display adapters because it does not have any standard socket any more (the old ones had at least DVI), only display port.
  • Just for the list: the power adapter is designed perfectly, plus every adapter fits every Mac, thus forgetting your’s at home is not such a big thing! That’s great, no doubt about it.

Usability Mistakes

  • Cmd+W closes a window (e.g. a browser tab), Cmd+Q quits the entire application… The keys for Q and W are just exactly next to each other on every keyboard and therefore it’s easy to hit Cmd+Q accidentally. Saidly it’s the same for Linux and similar for Microsoft Windows: Ctrl+F4 vs. Alt+F4.
  • The “cut-and-paste” paradigm is not supported (FYI, in an Apple user forum I read a justification as follows: "Imagine you cut a file, continue working and finally forget you cut it. Then when you shutdown your computer… the file will be lost!" :D)
  • Making a window fullscreen size can be very tedious because of two reasons: the "+" button in the upper left does not maximize the window horizontally (it gets widen to a width OSX "considers" reasonable) plus there is only a single resize grip on each window in the lower right. Thus, it requires precise moving and scaling of the window to get it fullscreen – instead of having just a button to click – or double clicking the title bar of the window to maximize it as usually supported in other OS.

One question remains: With respect to financial success, is UX more important than usability and utility?
According to this list and the success of Apple (though it seems the peak has been passed) the answer is yes.
Also, this list is not meant to badmouth Apple in general, I still consider Apple’s "computing products" (including Mac Books, iPhone, and iPad) to be some of the best hardware-software combinations available today, though, I definitely prefer my CrunchBang for being super fast and slim, efficient, highly configurable, completely controllable via the keyboard, and open source! 🙂

p.s. I stopped using OSX at version "Snow Leopard" at the end of 2012 so I might miss the latest features.

Scale images for the web — quickly

Starting Gimp just to scale down some photos to send them by mail? Meh.
How about mark them, right-click, "Send To" -> "Scale for web"?
No problemo:
First, get a tool to scale images:

sudo apt-get install imagemagick

Now, create a script that takes a list of images as input, scales them down, and stores them with a prefixed file name. Go!

#!/bin/bash
pwd > ~/f0.txt
echo "$@" > ~/f1.txt
for file in $@
do
	convert "$file" -quality 50 -resize 1024x768 "${file%/*}/web-${file##*/}"
done

Make sure we chmod +x the script file we just created.
Next, let’s integrate it into "Send To". (This is an example for Thunar)
Create a .desktop file in the sendto folder, e.g. /usr/share/Thunar/sendto/scale-for-web.desktop with following content:

# Scale images for the web
[Desktop Entry]
Type=Application
Version=1.0
Encoding=UTF-8
TryExec=scale-for-web
Exec=scale-for-web %F
Icon=
Name=Scale for the web
MimeType=image/jpeg;image/jpg;image/png

Replace both scale-for-web with the name you gave to your script file. Is the script’s folder in $PATH? No idea? Then just put the entire path+filename.
Now, open Thunar, select some images (Jpeg or Png), right-click, "Send To" -> "Scale for the web", wait a few seconds and you’ll see web-....jpg files appearing in the same folder. 🙂

References: how to scale images with imageMagick, extract path segments, more details on string manipulation

Hash in JS

Generating hashes from strings is a very handy thing to have. Surprisingly, in JavaScript, an arithmetic-operation-based algorithm is about *three times* faster than one using bit-wise operators–see for yourself. I suggested to add the most performant method as hash method to underscore.string.js.

So much about statements on stackoverflow like

The hash << 5 - hash is the same as hash * 31 + char but a LOT faster.

(cf. here)

In times of interpreted languages and virtual machines one should reconsider before applying the same old rules again.

p.s. If you know about alternative algorithms/implementations, please add them to the test and leave a comment here, I'm curious to see what else there is. Btw. I was very surprised that the neat reduce-based version failed so miserably.

Mongo fails when starting meteor

If your are seeing this one when starting meteor:
Unexpected mongo exit code 100. Restarting.
Can't start mongod
MongoDB had an unspecified uncaught exception.

Check if mongoDB is running already, otherwise, if you can find a file called .meteor/local/db/mongod.lock means that mongo did not stop properly last time. Just deleting this lock file should fix it. It’s a bit unfortunate that the error message doesn’t give any hint in this direction.