Friday, November 16, 2007

More Spatio-Temporal Warping

Here are a couple more demo clips from the team that brought you Spatio-Temporal Warping.
In these clips, a video pan of a scene is converted to a panorama with all activity occuring at the same time.
To really enjoy it, be sure to watch both the before and the after video. Link is in the title, above.

2D Video to VR Environment and Characters

For the last few months, I've been thinking a lot about what I would ultimately do with the hours of video I have taken over the course of my children's lives so far. I can't imagine actually watching all of that video, or even trying to edit it down. I don't really want to experience it in a linear fashion. After all, I was there. And while I may have experienced it through the viewfinder of a camera, still - I was there.

I've concluded that what would be useful would be to have a computer watch the video for me!
The computer would interpret the video, identifying the time and place, capturing the environment, and digitizing the people, creating photo-realistic 3D avatars of them.
In the long run, these videos will eventually be incorporated into my own (augmented) memory. The people's behaviors would be catalogued and pattern-recognized to the point that realistic simulations of the people - at various ages - could be made. I could have conversations and interactions with those who were no longer with me.

I can see that, ultimately, the computer's AI will be sufficient to really interpret the videos as well as I could if I were watching them. It would generate new memories, very similarly to what would happen to me - my memory "refreshed" - if I were to watch them.

Of course the computers of today aren't quite there yet. They are only now able to recognize the environment well enough to drive at about 14 miles an hour.

I would really like to see this technology developed, and I would like to know as much about it as I can. I have started (in my mid-life now) reading up on projective geometry, C++ programming and so on so that I can do some hobbyist-level playing around with this technology. I look forward to the day when the tools start to exist that would enable me to begin tackling these piles of videos I have here at home.

Thanks to a fellow named Augusto Roman (thanks for the link!), I now have a tool that will get me started. It's called Voodoo. It's camera tracking software that will "watch" a video clip and create both a point cloud of what is in the video, and a camera path. Both of these things can be exported to Blender, a free (and quite powerful) 3D animation package. I am now planning to try this with a few video clips. Of course, the package assumes a static scene with just the camera moving (if I understand it correctly), so I know it (by itself) isn't going to get me to the goal of separating out the moving objects from the environment. But it is a start.

Voodoo Camera Tracker is at:

Wednesday, September 26, 2007

Lunar - Hybrid Awaken

I have been listening to some wonderful music by an artist known as "Lunar". His whole album is available FREE for download, or you can buy a CD for 12 bucks. You can stream the music right from the website (link is in title, above).
If you like the music, you will want to make a contribution to the artist. I did!

Friday, September 07, 2007

Spatio-Temporal Video Warping

Check out the videos on this academic page (link in title).
These investigators are inventing new tools for navigating the 3 dimensional data cube of a video stream, and rearranging it spatio-temporally. You will see a demonstration of several time-warping effects. In one, a panoramic video is created out of a pan of a waterfall. In the panorama, the water falls at the same time, even though the imagery was taken at different times! In another demo, the demolition of a coliseum is shown, and in different versions of the demolition, different parts of the structure give way in different sequences. In yet another, the investigators can choose who wins in a swim meet. Check it out!

Monday, August 13, 2007

Sentient Rights in Accelerando

Here's an interesting quote from Charles Stross's book, "Accelerando."
"How you got here:
The center of the solar system - Mercury, Venus, Earth's Moon, Mars, the asteroid belt, and Jupiter - have been dismantled, or are being dismantled, by weakly godlike intelligences. [NB: Monotheistic clergy and Europeans who remember living prior to 1600, see alternative memeplex "in the beginning."] A weakly godlike intelligence is not a supernatural agency, but the product of a highly advanced society that learned how to artificially create souls [late 20th century: software] and translate human minds into souls and vice versa. [Core concepts: Human beings all have souls. Souls are software objects. Software is not immortal.]
Some of the weakly godlike intelligences appear to cultivate an interest in their human antecedents - for whatever reason is not known. (Possibilities include the study of history through horticulture, entertainment through live-action role-playing, revenge, and economic forgery.) While no definitive analysis is possible, all the resimulated persons to date exhibit certain common characteristics: They are all based on well-documented historical persons, their memories show suspicious gaps [see: smoke and mirrors], and they are ignorant of or predate the singularity [see: Turing Oracle, Vinge catastrophe].
It is believed that the weakly godlike agencies have created you as a vehicle for the introspective study of your historical antecedent by backward- chaining from your corpus of documented works, and the back-projected genome derived from your collateral descendants, to generate an abstract description of your computational state vector. This technique is extremely intensive [see: expTime-complete algorithms, Turing Oracle, time travel, industrial magic] but marginally plausible in the absence of supernatural explanations.
After experiencing your life, the weakly godlike agencies have expelled you. For reasons unknown, they chose to do this by transmitting your upload state and genome/proteome complex to receivers owned and operated by a consortium of charities based on Saturn. These charities have provided for your basic needs, including the body you now occupy.
In summary: You are a reconstruction of someone who lived and died a long time ago, not a reincarnation. You have no intrinsic moral right to the identity you believe to be your own, and an extensive body of case law states that you do not inherit your antecedent's possessions. Other than that, you are a free individual.
Note that fictional resimulation is strictly forbidden. If you have reason to believe that you may be a fictional character, you must contact the city immediately. [ See: James Bond, Spider Jerusalem.] Failure to comply is a felony."

I like the idea of a future AI that mines all historical documents, and recreates people based on them. As I have said, I'm hoping for a day when, either through augmentation, or in the course of being uploaded, my own mind can be fully mined (get it? my mind mined) for trace memories that can form the framework of a fully interactive memory space that allows me to revisit my parents, childhood friends, and family members as they were in those times. The AI will collate and index the memories, and may use logical processes to fill in my memory gaps.
In "Accelerando" though, something else is going on. The AI creates people for whom no neural structure is available. It's mining external documents - birth records, publications, the paper trail an entire life leaves behind - and recreating personality from those traces. There's even concern that some of these people might be fictional characters. These constructs believe they are the actual people they are modeled on. The society receiving these people, in the form of data packets, feel morally compelled to instantiate them, rather than just filing them in a memory store.
In today's world there are people reaching the end of their natural lifetimes who elect to have their heads - or even their whole bodies - held in stasis against decay in a bath of liquid nitrogen. Assuming that they can be reconstructed from this material - and I, for one, am almost certain they can - whose responsibility will it be to bring them back? Do they have a claim to a savings policy if one was set up for them? For they have just gone through a period in which they had no property rights whatsoever. This may be a moral question for us in the not-too- distant future.

Thursday, August 09, 2007

Of a Singular Mind

I seem to be in a very singular mood these days. I've been listening to a bunch of podcasted talks from the Singularity Summit - and elsewhere - during my bus commutes from Lompoc to Santa Barbara and back. One of my favorites is Bruce Sterling's 2007 South by Southwest rant. It's a terrific talk that he begins by not inviting everyone to his home in Austin for a keg party. That's after he reveals that he's been able to turn off the Wi-Fi on his laptop, so that he won't be tempted to blog his own talk while he's giving it!
I also ordered William Gibson's new book "Spook Country," and it has arrived, along with Kurzweil's book, "The Singularity is Near" - which I've already read as a library book, but which I think is worth reading again, this time with the freedom to make notes in the margins.
But I am denying myself the immediate pleasure of reading Gibson's book, for a couple of reasons: first, I had to finish rereading William Stross's book, "Accelerando." Originally I was looking for a certain passage in it that I thought would make an interesting blog topic, so I downloaded the full text, which, in the spirit of the FSF, is available for free. I grepped for the passage I was looking for, but then I started to just read. So I printed out about the first 30 pages or so, as something to read on the bus. I devoured that, and when I got home, I dug up my paperback copy of the book, and stashed it in my backpack. I just finished reading it yesterday, but I still can't read "Spook Country." I picked up Gibson's previous book, "Pattern Recognition," which I've also read before, but, after reading a review of "Spook," wanted to have fresh in my mind. I just started reading it last night.
So, it will be a few days to a couple of weeks before I can tackle "Spook Country." But by then I'll be fully prepared.
Meanwhile, I still need to pull that quote from "Accelerando."

Tuesday, August 07, 2007

Pas de Deux

I first saw the Norman McLaren film "Pas de Deux" in high school in the mid 1970s. It inspired me then, and it still inspires me.
McLaren worked for the National Film Board of Canada. What a gig! They really supported his work.
Definitely check out the link above. It's buried in the post title.

Tuesday, July 24, 2007

Photo Tourism demo

Here's a Java demo of Photo Tourism, the predecessor to Photosynth. The drag with Photosynth is that the demo only runs on Windows XP and Vista. Bleccch. Here, all you need is a Java-enabled browser (I'm using Firefox).

Things to do

There is a multitude of things I don't understand, that I would like to. I want to move along in converting my photo and video archives into a VR memory-assist library or museum. Initially, this VR space will occupy a computer memory. Eventually, I hope that it will be incorporated into my own augmented brain. This seems to be the subject this blog is taking, somewhat without my having planned it that way.
I have begun to realize that, in a way, EVERYTHING is software. That means that without the personal ability to create and modify software, I am without a very necessary skill set. I have begun to study the C++ programming language. I downloaded the free text of "Thinking in C++" and I am studying that every day. In addition, I have discovered many articles about how various groups have implemented the conversion of video streams into panoramic mosaics and even full 3D environments. Imagine being able to video record a scene, and then feed that video into an application that converts it into a 3D space. That's what is coming. If you are interested, you should look up the Photosynth website. This is a Microsoft Research product that correlates any number of photographs of an area into a 3D space, and retains the ability to navigate to each image. The demo video is really cool.


A great video showing a demonstration of Photosynth - a program or database that does some of what I have been talking about

Monday, June 11, 2007

Convert Video to VR

I thought of a neat piece of software I would like to see:
This application would extract the background from a video stream that included pans and tilts in the shot.
Say you have a shot of someone running across a field. The software would process the shot into a VR background image. Using pattern recognition, each frame would add to the background image as the camera panned across a new region. Only the background would survive this process. If there was an object being tracked in the shot, then this object would not be part of the background "plate".
An extension to this process would be the ability to derive a new view of the object moving across a static background. It would be possible to view the object from a select range of angles. This part would be a little harder if the original object was near enough to the camera so that the camera saw multiple sides of the object. But this could all be sorted out.
This idea came out of my thoughts of how my video and image collection could be used to create a virtual world for myself. Sometime in the future, all of my media will be digitized and used to create a space for me to explore my life's history. People from my past will be recreated using AI algorithms and every bit of video of that person. The AI algorithm would be able to simulate the person, so that person would live on in my memory.
Eventually this data will be integrated with my own memories. Even more, my memories will be mined by the software to help me recollect things that are only dim memories for me now.
Then that first kiss, the bee sting I got when I was three - and so on - would be as real as new.