Last week Apple filled a series of seven major patent filings directly related to advancing camera features that will be found in future iterations of Apple’s iDevices such as the iPhone and iPad. The new patent applications cover gesture techniques and in-depth detailing of their advanced red-eye technology. Real camera buffs will have a feast in learning about what Apple is working on and we list all of the patent links so that you could dive right in to learn about every little twist and turn. If anything – we know that Apple always updates the iPhone’s camera features each and every year like clockwork. Will any of Apple’s latest advancements make it to the iPhone 6? Only time will tell. And lastly, we take a brief look at a simple legal update to Apple’s original patent filing for Siri that doesn’t advance Siri in any way whatsoever.
1. Gesture-Based Configuration of Image Processing Techniques
The first camera related patent that we briefly cover generally relates to iDevice cameras that capture and display filtered images on a touch screen display.
With more complex image processing routines being carried out on personal electronic devices, such as graphically-intensive image filters, e.g., image distortion filters, the number and type of inputs, as well as logical considerations regarding the orientation of the device and other factors may become too complex for client software applications to readily interpret and/or process correctly. Additionally, if the image that is currently being displayed on the device has been distorted via the application of an image filter, when a user indicates a location in the distorted image to base the setting of auto exposure, auto focus, and/or auto white balancing parameters upon, additional processing must be performed to ensure that the auto exposure, auto focus, and/or auto white balancing parameters are being set based on the correct underlying captured sensor data.
Apple’s patent abstract for patent application 20120242852 states in-part that their invention pertains to apparatuses, methods, and computer readable medium for mapping particular user interactions, e.g., gestures, to the input parameters of various image filters, while simultaneously setting auto exposure, auto focus, auto white balance, and/or other image processing technique input parameters based on the appropriate underlying image sensor data in a way that provides a seamless, dynamic, and intuitive experience for both the user and the client application software developer..
About Apple’s Patent Figures: FIG. 3 illustrates a user interacting with a camera device via a touch gesture, in accordance with one embodiment; FIG. 4 illustrates a user tap point and a typical exposure metering region on a touch screen of a camera device; FIG. 5A and FIG. 5B illustrate an exposure metering region that has been translated based on an applied image filter that Apple calls a “shrink filter”; FIG. 8 illustrates a user tap point and corresponding relevant image portion on a touch screen of a camera device, in accordance with one embodiment; FIG. 9 illustrates a light tunnel image filter effect based on a user tap point on a touch screen of a camera device
2. Assisted Image Capture
Apple’s second camera invention found under patent application 20120242849 pertains to systems, methods, and computer readable medium for assisting the user of a digital image capture device take a well composed image.
In general, techniques are disclosed herein for stabilizing an image in the digital image capture device’s (e.g., preview) display so as to indicate to the user that the image is well composed–thereby assisting the user in capturing quality images. Once stabilized, a selected point or location in the image doesn’t appear to move in the display for small motions of the device itself. If the camera is moved more than a small amount, the display is unfrozen whereafter the display tracks the view of the camera in a normal manner. In addition to stabilizing the displayed image, additional visual cues may be presented in the device’s display.
It is in this patent filing that Apple introduces to an image being able to be badged to further indicate to a user the image has been judged to be well composed. The term “badging” means to display a visual cue to the user that is not in the view itself – see patent figure 6A.
It should be noted that Apple’s IP filing specifically points to Processors 725 and 805 may include any programmable controller device including, for example, one or more members of the Intel processor families such as Atom, Core or others in addition to Core and ARM processors from ARM. This is the first time that I’ve seen Apple list the Atom or Core processor in an iPhone related patent. While the mere mention of the processor has no meaning today, the fact that Apple has chosen to legally mention it in their invention means that it’s a possible future candidate that should be listed in their filing.
3-7 Relate to Advancing Red-Eye Removal
Apple is declaring war on red-eye. On Thursday, Apple filed five major patent applications relating to the advancement of red-eye removal under numbers 20120243783, 20120243780, 20120242861,20120242681 and 20120242675
In photography, red-eye is the occurrence of glowing red pupils in a color photograph due to eye shine. Red-eye is believed to be caused by the red reflection of the blood vessels in the retina when a strong and sudden light strikes the eye. The tonality and intensity of red-eye may vary from person to person based on ethnicity, pigmentation levels, and other factors.
Apple is out to eliminate red-eye in your photos even though they currently have tools for eliminating red-eye in both iPhoto and Aperture. It’s very clear from Apple’s latest round of patents that they’re going deeper into the science behind this technology so as to further improve this feature in the future.
Apple’s IP filing generally relates to red-eye removal techniques, and more particularly, to specific characterization, discernment, and repair techniques utilizing multiple recognition channels (e.g., red, golden, and white recognition channels).
Technically speaking, Apple’s patent abstract states that their invention relates to apparatuses, methods, and computer readable media for red-eye removal techniques using multiple recognition channels. In the following examples, red, golden, and white recognition channels are used. A recognition channel is the monochrome extraction from a color photograph in a manner designed to make one kind of red-eye artifact glow with maximum contrast. Once the red-eye artifact has been characterized by, e.g., size and location, the techniques disclosed herein may then discern whether the red-eye artifact is, for example, a red-, golden-, or white-eye case by examining the configuration and characteristics of prominence bitmasks created for the various recognition channels. Once the type of red-eye case has been discerned, the techniques disclosed herein may then replace the artifact with a photographically reasonable result based on the type of red-eye case being repaired. Specular reflection may also be re-added to the photograph.
About Apple’s Patent Figures: FIG. 3 illustrates a photo on a camera device’s preview screen having red-eye artifacts; FIG. 3B illustrates a user interacting with a camera device via a touch gesture; FIG. 6 illustrates the red, golden, and white recognition channels corresponding to a red-eye artifact in an image; FIG. 9A illustrates the connected components of a prominence bitmask overlaying the underlying image, in accordance with one embodiment; and finally, FIG. 9B illustrates the connected components of a prominence bitmask without the underlying image underneath.
Apple Legally Updates Siri
On Thursday Apple filed a continuation patent for Siri. Patently Apple doesn’t post continuation patents as separate reports so as to not waste your time. This is what we usually post on the bottom of an original patent application report when it’s appropriate: “Generally speaking, continuation patents represent tweaks made to patent claims in an effort to get the patents granted by the USPTO and don’t represent any noteworthy new development from the original patent filing.” To clarify, the last line relates to nothing noteworthy for consumers.
There were 47 patent figures in the original patent filing and there are 47 in the continuation patent. The patent abstracts are identical as are their patent backgrounds and patent summaries. To check it out for yourself, here’s the January filing vs. the September filing.
So what was the big deal in this week’s continuation patent? Well, exactly what we continually state about continuation patents: “continuation patents represent tweaks made to patent claims in an effort to get the patents granted by the USPTO.” Where were the tweaks in this latest Siri filing update?
Apple’s original patent lists the words “elicit, eliciting, elicitation” over 100 times and the continuation patent has removed any references to such terms or expressions. Additionally, Apple added “speech-based conversational interface” in three of their claims 1, 7 and 13. Obviously Apple Legal decided this change was legally required in order to get the patent filing approved by the US Patent and Trademark Office – just like we point out in context with continuation patents. For the general public or consumer, such tweaks are irrelevant.