Html5 video puzzle – proof of concept

Written by Alex Wolkov


I’ve decided to learn more about html5, since Google and Apple and a lot of these BIG companies say it’s the future.

I went to http://canvasdemos.com and was amazed that all this can be achieved with canvas tag.

I specifically liked the blowing up video into pieces , check it out, it’s AWESOME.

Sean from craftymind.com has done and amazing proof of concept, you should really check it out. I played with it and then realized, that clicking is the only interactivity there is and it seems that there’s more there. much more.

Here’s what I came up with :

Click on Randomize Pieces to explode the video to pieces and Begin assembling.

demo2 has video courtecy of bigcbuckbunny movie and lots more pieces (use chome)

I’ve decided to try and make something using the drawImage() method for the canvas, and inserting the video feed.

Goals :

  • Learn a bit more about ,html5 ,  video encoding
  • Use tools I know to interact with the new stuff (jquery, css)
  • Create something awesome!
  • Make it fun

The idea:

The idea was simple, take Sean’s example, and add interactivity, first I was leaning towards making a “pyatnashki” ор fifteenpuzzle puzzle game, only using video, but then I decided it’s cool to leave it as a regular puzzle without constraints.

The actual work:

First I had to use some movie, it had to loop good enough, since as it turned out later, video puzzle isn’t really easy. I’ve used video shot by my uncle I offcourse had to format it properly, well it turns out, right now there are weird ways to input

I’ve re-encoded the video to 2 files ( .ogv and .mp4) using this tutorial : http://diveintohtml5.org/video.html

I then had to choose what will I do the actual interactivity with, the canvas itself, or DOM elements. I’ve chosen DOM elements, that maybe slower, but it’s a diferent approach then I’ve seen, plus I already know how to manipulate DOM, and can use that to my advantage.

I’ve created a factory function, that picks up on original video width and height, and takes the number of pieces that I want, and creates the necessary amount of canvas elements.

I then use drawImage each frame of the video, and copy pixels to each canvas according to their location on the matrix.

  • Pingback: Alex Wolkov Makes a HTML5 Video Jigsaw Puzzle « Awesome But Useless

  • Hamid Raza

    awesome

  • Fabio Gasparri

    Excellent work!
    I was playing around some similar ideas, and one of the things I found when doing some tests is that trying to draw a video on a canvas ( canvas.drawImage(video,0,0) ) works perfectly fine in Chrome and Firefox, but unfortunately (and surprisingly), it doesn’t work on my iPad.
    Do you have any idea or thoughts on how to solve this?
    Thanks!

  • Altryne

    Hi Fabio,
    Thanx for the comment, I noticed the same thing, video just doesn’t draw i canvas on the ipad whatever I did :(
    Too bad because thiw whole puzzle idea was originally for the ipad.
    Its a “mobile safari” thing, and there are bugs on this in bagtrack
    Anyway, I later stumbled upon svg, and saw somewhere that you can also put video on svg, which the mobile safari supports,
    Just need to try and see if that’ will work

  • Fabio Gasparri

    Hi Altryne, thanks for you reply. Not sure I get your idea, but I will give it a try. you say to put the video tag inside an svg object, an then get the video frames from the svg object?

  • http://www.iiistudio.com James Griggs

    Hi Altryne,
    Were you able to get this to work on the iPad using svg? Did the iOS 4.2 update make this work for canvas?
    Very interested how far you got with this.
    Sincerely,
    James

  • Altryne

    unfotumately I havent had time to… Ive tried and failed back when iPad was on 3.2

    now that it’s on 4.2 I can try and see if they updated something, and ‘ll update my blog.

  • http://www.iiistudio.com James Griggs

    Thanks Altryne, I would really love to know if it is possible. Are you available for contract work?

  • Alexander

    Thanks. Good step.
    In the past, there was “Alive! jigsaw” which turns the images and video files to jigsaw. The freeware v4.12 requires indeo codec and works with any jpg or avi file if you put these files in folder of the program, and then the program will detect them and you can choose any one. later, this game became shareware but you can find the freeware V4.12 from some of websites. I want to play your game offline in html5 web prowser as free open source alternative with more freedom to the user to choose own files. Please, try and put a roadmap for the next editions of the game.

  • Brad

    Hi,
    The problem I am having is the following:
    I use a video and a canvas tag and capture a frame from the video tag and place it into the canvas tag, but when I create a reference of the canvas in javascript so that I may return the dataURL it bombs out.
    Here is the code I use to place the frame from the video:
    _________________________________________________________________
    var video = document.getElementById(“video”);

    var canvasDraw = document.getElementById(‘imageView’);
    var w = canvasDraw.width;
    var h = canvasDraw.height;
    var ctxDraw = canvasDraw.getContext(’2d’);

    ctxDraw.clearRect(0, 0, w, h);
    ctxDraw.clearRect(0, 0, w, h);
    ctxDraw.drawImage(video, 0, 0, w, h);
    _________________________________________________________________
    The above works perfectly.
    Below is the code to get the dataURL:
    _________________________________________________________________
    function getURIformcanvas() {
    var ImageURItoShow = “”;
    var canvasFromVideo = document.getElementById(“imageView”);
    if (canvasFromVideo.getContext) {
    var ctx = canvasFromVideo.getContext(“2d”); // Get the context for the canvas.

    var ImageURItoShow = canvasFromVideo.toDataURL(“image/png”); //<– It fails on this line.
    }
    var doc = document.getElementById("txtUriShow");
    doc.value = ImageURItoShow;

    }
    _______________________________________________________________
    It always fails on the line:
    var ImageURItoShow = canvasFromVideo.toDataURL("image/png");

    Any thought on what might be the problem. If I load a normal image into the canvas it works fine, but as soon as I load the image from video into the canvas that line fails.

    Any ideas?

  • Pingback: multi-streamed video puzzle to showcase a network protocol. Ehm… howto? | Done Is Better Than Perfect

  • Dane

    were you ever able to get this kind of thing to work on an ipad? i have been getting “tainted canvas” errors, but only on mobile devices, not on desktop, and even though my videos are on the same server as the html page.

  • altryne

    hey, sorry for the super late reply.
    Nope, unfortunately, drawImage isn’t implemented on mobile devices for some reason :(