MOTION ESTIMATION USING COMPLEX DISCRETE WAVELET TRANSFORM

Transkript

1 MOTION ESTIMATION USING COMPLEX ISCRETE WAVELET TRANSFORM A THESIS SUBMITTE TO THE GRAUATE SCHOOL OF NATURAL AN APPLIE SCIENCES OF THE MILE EAST TECHNICAL UNIVERSITY BY HÜSEYİN SARI IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE EGREE OF MASTER OF SCIENCE IN THE EPARTMENT OF ELECTRICAL AN ELECTRONICS ENGINEERING SEPTEMBER 3

2 Approval of the Graduate School of Natural ad Applied Sciece Prof. r. Caa Özge irector I certify that thi thei atifie all the requiremet a a thei for the degree of Mater of Sciece. Prof. r. Mübeccel emirekler Head of epartmet Thi i to certify that we have read thi thei ad that i our opiio it i fully adequate i cope ad quality a a thei for the degree of Mater of Sciece. Prof. r. Mete Severca Supervior Examiig Committee Member Aoc. Prof. r. Gözde Bozdağı Akar Prof. r. Mete Severca Aoc. Prof. r. Aydı Alata Prof. r. Göül Turha Saya r. Uğur Murat Leloğlu ii

3 ABSTRACT MOTION ESTIMATION USING COMPLEX ISCRETE WAVELET TRANSFORM Sarı Hüeyi M.Sc. epartmet of Electrical ad Electroic Egieerig Supervior: Prof. r. Mete Severca September 3 8 Page The etimatio of optical flow ha become a vital reearch field i image equece aalyi epecially i pat two decade which foud applicatio i may field uch a tereo optic video compreio robotic ad computer viio. I thi thei the complex wavelet baed algorithm for the etimatio of optical flow developed by Magarey ad Kigbury i implemeted ad ivetigated. The algorithm i baed o a complex verio of the dicrete wavelet traform CWT which aalyze a image through block of filterig with a et of Gabor-like kerel with differet cale ad orietatio. The output i a hierarchy of caled ad ubampled orietatio-tued ubimage. The motio etimatio algorithm i baed o the relatiohip betwee tralatio i image domai ad phae hift i CWT domai which i atified by the hiftability ad iterpolability property of CWT. Optical flow i etimated by uig thi relatiohip at each cale i a coare-to-fie hierarchical maer where iformatio from fier cale i ued to refie the etimate from coarer cale. The performace of the motio etimatio algorithm i ivetigated with variou image equece a iput ad the effect of the optio i the algorithm like curvature-correctio iterpolatio kerel betwee level ad ome parameter value like cofidece threhold

4 maximum umber of CWT level ad miimum fiet level of detail are alo experimeted ad dicued. The tet reult how that the method i uperior to other well-kow algorithm i etimatio accuracy epecially uder high illumiace variatio ad additive oie. Key word: Stereo viio phae baed motio etimatio complex dicrete wavelet traform gabor filter ubbad tree decompoitio. iv

5 ÖZ KARMAŞIK AYRIK ALGACIK ÖNÜŞÜMÜ KULLANARAK HAREKET KESTİRİMİ Sarı Hüeyi Yükek Lia Elektrik ve Elektroik Mühediliği Bölümü Tez Yöeticii: Prof. r. Mete Severca Eylül 3 8 Sayfa Optik akış ketirimi özellikle o yirmi yılda ıralı görütü aalizi kouuda çok öemli bir araştırma alaı halie gelmiş ve tereo optik video ıkıştırmaı robotlar ve bilgiayarla görme gibi koularda uygulama alaları bulmuştur. Bu tezde Magarey ve Kigbury i optik akış tahmii içi geliştirdiği karmaşık ayrık dalgacık döüşümü tabalı algoritma araştırılmış ve uygulamıştır. Algoritma remi farklı ölçek ve yöelimlerdeki Gabor bezeri üzgeç kullaıla bloklarda geçirerek aaliz ede ayrık dalgacık döüşümüü komplek bir veriyoua dayaır. Çıktı ölçeklemiş ve altöreklemiş yöelim-akortlu altreimleri ıradüzeide oluşmuştur. Hareket ketirim algoritmaı karmaşık ayrık dalgacık döüşümüü aradeğerledirilebilirlik ve kaydırılabilirlik özelliği tarafıda ağlaa reim alaıdaki değişimler ve döüşüm alaıdaki faz kaymaları araıdaki bağıtıya dayaır. Optik akış her ölçekte bu bağıtıyı kullaarak düşük çözüürlükte yükek çözüürlüğe gide ve yükek çözüürlükteki verileri düşük çözüürlükteki verileri rafie etmek içi kullaıldığı bir matıkla tahmi edilir. v

6 Hareket ketirim algoritmaıı performaı farklı reimler girdi olarak kullaılarak araştırılmış ve eğrilik düzeltmei düzeyler araı aradeğerleme tipi gibi modifikayoları ve güveilirlik eşik değeri makimum eviye ve e detaylı çözüürlük gibi parametreleri etkii aaliz edilmiştir. Tet ouçları göteriyor ki metod iyi bilie diğer algoritmalara orala özellikle yükek aydılatma değişimleri ve gürültü eklemei koşulları altıda çok daha iyi ketirim doğruluğu vermektedir. Aahtar kelimeler: Çift kaallı görme faza dayalı hareket ketirimi karmaşık ayrık dalgacık döüşümü gabor üzgeçleri altbad ağaç ayrışımı. vi

7 ACKNOWLEGEMENTS I would like to thak my upervior Mete Severca for hi uflaggig upport ad expert guidace throughout the developmet ad improvemet of thi thei. I of coure would like to thak my paret for beig ad upportig me i educatioal life. I am alo grateful to Mahir Özdemir for hi iightful advice friedly upport ad providig upport o Matlab. I would alo like to expre my deep gratitude to all thoe who have upported ad ecouraged me i completig thi thei. vii

8 TABLE OF CONTENTS ABSTRACT...III ÖZ V ACKNOWLEGEMENTS VII TABLE OF CONTENTS..VIII LIST OF TABLES.X LIST OF FIGURES..XI LIST OF ABBREVIATIONS...XIII CHAPTER. INTROUCTION. STEREO VISION..... THE MOTION ESTIMATION PROBLEM APPROACHES TO THE MOTION ESTIMATION PROBLEM..4.4 THE CWT MOTION ESTIMATION ALGORITHM..5.5 THESIS OUTLINE..5. REVIEW OF MOTION ESTIMATION TECHNIQUES MOTION ESTIMATION TECHNIQUES..7.. GRAIENT-BASE TECHNIQUES... BLOCK-MATCHING TECHNIQUES 3..3 FREQUENCY OMAIN TECHNIQUES...5. SUMMARY MOTION ESTIMATION BASE ON COMPLEX-WT THE COMPLEX ISCRETE WAVELET TRANSFORM ONE-IMENSIONAL WAVELET TRANSFORM CONTINUOUS TIME ISCRETE TIME WAVELET TRANSFORM AS A FILTER BANK TWO-IMENSIONAL ISCRETE WAVELET TRANSFORM COMPLEX WT ONE-IMENSIONAL CWT TWO-IMENSIONAL CWT MIRROR FILTERS CONTRIBUTING REGIONS 3 viii

9 3..4 SUMMARY MOTION ESTIMATION ALGORITHM SINGLE LEVEL ESTIMATION SUBBAN SQUARE IFFERENCES CWT COEFFICIENTS INTERPOLATION AN POSITION SHIFTABILITY QUARATIC SURFACES HIERARCHICAL ESTIMATION INTERPOLATION AN SCALING OF QUARATIC SURFACES CUMULATIVE SQUARE IFFERENCES COARSE TO FINE APPROACHES CURVATURE CORRECTION CONFIENCE MEASURE FILTERS COEFFICIENTS AN CORRESPONING GABOR PARAMETERS SUMMARY SIMULATIONS TEST ATA ERROR MEASUREMENT TESTING THE ALGORITHM CONFIENCE THRESHOL NUMBER OF PYRAMI LEVELS J MAX CHOICE OF FINEST LEVEL J MIN FIEL INTERPOLATION TYPE TYPE OF GABOR FILTERS CURVATURE CORRECTION ILLUMINANCE VARIATION NOISE IMMUNITY SUMMARY CONCLUSIONS & FUTURE WORK CONCLUSIONS FUTURE WORK...67 REFERENCES.. 68 ix

10 LIST OF TABLES TABLE 4.. Mea error agle ad field deity calculated for variou cofidece threhold. 4.. Mea error agle for tree equece mi fixed to 3 uig biliear iterpolatio Mea error agle for tree equece max fixed to 5 uig biliear iterpolatio with cofidece threhold et to Mea error agle of tree equece uig taircae ad biliear iterpolatio Mea error agle for 4-tap RI pair ad 8-tap Gabor filter with max =5 mi = cofidece threhold =.95 uig biliear iterpolatio Mea error agle of tree equece for variou e t value Mea error agle of tree ad Yoemite equece MSE of tree ad Yoemite equece. x

11 LIST OF FIGURES FIGURE.. Model of huma viual ytem.. Model of obtaiig -d proectio of a 3d-cee.. OFC cotrait lie with poible olutio v a vb ad v c.. The aperture problem.3. Block-matchig earch algorithm 3.. Two-bad buildig block for dyadic WT. 3.. Subbad decompoitio tree 3 level 3.3. Buildig block for eparable WT o image I Partitio of the two-dimeioal uit frequecy cell by a igle-level eparable WT 3.5. Elliptical quai-circular cotour of the magitude repoe of the twodimeioal CWT wavelet filter at level ad Two-dimeioal CWT level 3.7. The magitude repoe of the mirror filter coverig the ecod quadrat of the uit frequecy cell Frequecy repoe of ubbad filter i -d CWT decompoitio Hierarchical tructure of CWT-baed motio etimatio algorithm Cotour of urface S =...6 ad their um S cetre bottom for a typical diplacemet etimatio. 3. Miimum lie of quadratic urface give by Expreio 3.5 a whe the motio i well-defied the miimum lie correpodig to the differet orieted ubbad cloely itercept each other b whe the motio i ot well-defied. 3. Choice of miimum locatio cloet to the origi i cae whe S i degeerate. 4.. acurret frame of ythetic tree equece b Curret frame of Yoemite equece xi

12 4.. True motio field of ythetic equece a Tralatig Tree b ivergig Tree c Rotatig Tree d Yoemite 4.3 Mea error agle ad field deity plot for divergig tree image at level. 4.4 Mea error agle plot of tree equece for variou e t value. a ivergig tree b Tralatig tree c Rotatig tree 4.5 Curvature correctio to tralatig tree equece uig 8-tap Gabor filter 4.6 Etimated motio field of tree ad Yoemite equece at 8-pel reolutio with cofidece trehold =.95 = max 5 = mi uig 4-tap RI pair biliear iterpolatio with curvature correctio. 4.7 Mea error agle agait uiform iteity perturbatio aadditive offet b Scalig 4.8 Mea error agle uder additio of white Gauia oie xii

13 LIST OF ABBREVIATIONS CWT: Complex icrete Wavelet Traform CS: Cumulative Squared ifferece F: iplaced Frame ifferece R: iplaced Regio ifferece WT: icrete Wavelet Traform FIR: Fiite Impule Repoe FT: Fourier Traform MA: Maximum Abolute ifferece ME: Motio Etimatio MSE: Mea Square Error OFC: Optical Flow Cotrait QMF: Quadrature Mirror Filter RI: Rotatio Ivariat SS: Subbad Squared ifferece STFT: Short Time Fourier Traform WT: Wavelet Traform xiii

14 CHAPTER INTROUCTION Huma ad aimal aturally develop the ability to dicer obect acertai their motio ad avigate i three-dimeioal pace to iteract with their urroudig. All of u are equipped with a pair of eye ad perform viual fuctio uch a depth perceptio obect recogizig trackig etc. through the ue of thi tereoviio mechaim. Although thi effective viual perceptio mechaim ha ot yet bee fully explaied or matched by ay artificial viio ytem the idea of icorporatig uch viio i machie gave rie to ivetigatio ad implemetatio i europhyiological ad computer viio area epecially i the pat two decade. Numerou tudie i both area have how that motio i a baic data for viual fuctio. I fact the aalyi of a equece of image provide iformatio about the threedimeioal hape ad tructure of obect the relative depth betwee differet obect or their traectorie ad directio of motio which could ot be obtaied from a igle image. Therefore reearch work o motio etimatio ha already bee carried out to variou applicatio i which a compact repreetatio of the chage betwee the ucceive frame i a digital image equece i required. Oe applicatio i the obtacle detectio ad avigatio i robotic. Aother applicatio i the egomotio recovery i.e. etimatio of the oberver motio. Alo much work ha alo bee geerated i the field of video codig ice compreio of image equece uig motio etimatio reduce the badwidth requiremet for a efficiet tramiio. It alo fid place i medicie ad meteorology by proceig related image. Aother area of high importace i the detectio ad trackig of movig target i military applicatio. I the pat decade wavelet ha become a importat aalyi techique for motio etimatio. The ucce of wavelet aalyi whe applied to motio etimatio problem ha how it uitability a a approach to the tereoviio problem. The work preeted here i

15 cocered with motio etimatio baed o wavelet aalyi developed by J. Magarey ad N. Kigbury [].. Stereo Viio Stereo viio i related with the recovery of the three dimeioal hape of a cee baed o two image take of that cee from lightly differet viewpoit. Huma viio ytem ca be modeled a two camera eparated by a cotat ditace ofte referred to a a tereo head. Figure. Figure. Model of huma viual ytem For the proce of obtaiig tereo image pair the commo method i to ue two camera diplaced from each other by a kow ditace. Aother method i to ue a movig camera. Whe the eor itelf i movig motio detectio aloe i ot eough. The eor motio geerate a apparet motio of the backgroud which eed to be compeated or egmeted from the target motio i order to allow detectio of motio. I each cae the oly detectable motio i a image equece follow from the patio-temporal chage of the gray level iteity fuctio of the equece kow a the optical flow. A expected ecod image i ot the purely hifted verio of the firt image. Thi i becaue the chage ca be caued by the relative motio betwee the chagig three dimeioal cee ad imagig device but ca alo be produced by chage i the image formatio proce itelf uch a illumiatio chage or oie i the electroic. Here the work will be o tereo image pair which are take from tatioary camera i.e. idetifyig true motio vector. If deired the thi true motio vector ca be ued to recotruct depth iformatio [ 3] ice the ditace betwee two image ource i kow.

16 . The Motio Etimatio Problem The motio etimatio problem deal with the aalyi of motio i two-dimeioal digital image which are the aphot of a three-dimeioal movig cee take at ucceive itat of time a how i Figure.. Figure. Model of obtaiig - proectio of a 3-cee I the ideal cae ay two-dimeioal poit i a image may correpod to aother twodimeioal poit i the followig image. Thi ca be formulated a u x + dx ˆ = u x. where x i the poitio vector of the pixel i the curret image dˆ x repreet the diplacemet vector of the pixel with x ˆ + dx repreetig the locatio of the pixel i the referece image ad { εz} u i the calar iteity fuctio of the image umber. But i real life there are a lot of factor effectig thi oe-to-oe correpodece i.e. additio of ome mea of oie to curret image. So the image equece i oberved to be oie corrupted which ca be formulated a I =. x u x + e x The frame are captured by a camera at a give frame rate ad each frame ca be viewed a a fuctio of the light iteity iformatio take by the camera at differet time itat. Sice the captured equece repreet a proectio of the three-dimeioal cee to two dimeio ay image i ot a purely hifted verio of the previou image ad the relatiohip betwee the obervable chage i image iteitie ad the motio of threedimeioal obect i the cee i complicated. Thi complicated relatiohip i caued by the lo of iformatio i the proce of proectio from three to two dimeio. Thi i 3

17 kow a the proectio ambiguity or the correpodece problem which mea that everal poit i the three-dimeioal cee with potetially differet motio may proect to the ame two-dimeioal poit i the image. There are alo other reao for thi complexity which add ome kid of oie to the motio etimatio proce. Thee are the effect like occluio illumiatio chage aberratio ad ditortio i the image acquirig optic etc. Aother iteretig factor i whether movig obect are rigid or chage hape durig the motio for example a milig huma face. Becaue of the lo of iformatio while proectig from three-dimeioal pace to twodimeioal pace the problem i ill-poed i.e. a uique olutio of the correpodece problem doe ot alway exit. Notice that a ditictio i ometime made betwee the true motio field which i the proectio of the relative 3 motio betwee cee ad camera ad the optical flow field which decribe the variatio of the graycale patter over pace ad time. Ideally they are idetical if the image formatio proce i free of illumiatio chage oie i electroic obect hape chage etc..3 Approache To The Motio Etimatio Problem The motio etimatio algorithm ca i geeral be claified ito two mai categorie. Thee are low-level ad high-level computer viio algorithm. High-level method are baed o extractig high-level feature of the image uch a edge corer obect boudarie or complete obect i order to olve the correpodece problem. The advatage of the feature-baed approache i that matchig oly o the bai of thee feature i relatively fat due to the mall umber of poit extracted from each image. Alo fale matche are relatively le becaue matchig i oly attempted for the poit which are eaiet to match. However the umber of fale matche for the feature baed approache icreae with a cee cotaiig wide featurele regio mooth urface or whe feature i a image are occluded i the other. Alo eve there are o occluio i the image a highly featured area alo icreae the umber of fale matche. Low-level approache ue low-level image decriptor like iteity of the image for motio etimatio proce. The low-level approache are alo referred a the optical flow etimatio approache which ca be categorized ito gradiet-baed block-matchig ad frequecy domai techique. Thi work referred to a the CWT motio etimatio algorithm will be o optical flow approache ad correpod to the latter category. 4

18 .4 The CWT Motio Etimatio Algorithm The CWT motio etimatio algorithm i baed o the complex dicrete wavelet traform developed by Magarey ad Kigbury [ 3 7]. I order to compute the motio field the algorithm ue the relatio betwee the phae of the traform coefficiet ad tralatio i the image domai. The choice of the algorithm from a wide variety of approache i becaue of three mai factor. The algorithm ue a dicrete wavelet traform implemetatio imilar to a method developed by Mallat[] which make ue of a efficiet pyramidal algorithm baed o covolutio with oe-dimeioal 4-tap filter. The baic filter pair ued i implemetatio may be modeled a Gabor filter which have a trog implicatio from a phyiological/biological poit of view. I fact there are variou ivetigatio o modelig the mammalia viio ytem with a preproceig tage i which Gabor filter cotitute the mai compoet. Motio etimatio i baed o the phae of the complex wavelet coefficiet. The phae-baed motio etimatio i relatively robut ad evolutio of cotat phae cotour of the complex filter output give a truer picture of the uderlyig patiotemporal tructure tha that of cotat iteity cotour which are ued i gradiet-baed method. Thi work decribe the ivetigatio ad applicatio of the CWT motio etimatio algorithm. The algorithm i implemeted i Matlab. Alo experimet are doe ad reult are extracted i order to compare the performace of the modificatio ad parameter chage i the algorithm..5 Thei Outlie The orgaizatio of the thei i a follow. Chapter preet a review of motio etimatio techique. It focue o low-level motio etimatio techique ad give detailed iformatio o them. Chapter 3 decribe the motio etimatio algorithm developed by Magarey ad Kigbury [] which i the bai of thi work. The algorithm i baed o a complex verio of the dicrete wavelet traform developed by Mallat []. Thi chapter i compoed of two mai part. The firt part decribe the complex dicrete wavelet traform tartig with a hort ummary of the wavelet traform i cotiuou ad dicrete domai the 5

19 implemetatio uig a quadrature mirror filter QMF bak ad fially the Gabor-like complex-valued FIR filter which are the bai of the CWT implemetatio. The ecod part i compoed of the decriptio of the motio etimatio algorithm itelf which ue the CWT coefficiet at variou reolutio i order to etimate motio i a coare-to-fie maer. Thi i doe by matchig the phae of the CWT coefficiet which are directly related to the diplacemet i the image domai. Chapter 4 i compoed of the imulatio aalyi ad reult with ythetic tree equece a iput of the implemeted motio etimatio algorithm. Alo ome optio are itroduced ad their cotributio to the performace of the propoed algorithm are aalyzed. 6

20 CHAPTER REVIEW OF MOTION ESTIMATION TECHNIQUES A metioed i the previou chapter there are everal method that ca be ued to etimate optical flow. Sice motio etimatio i a reearch area for early two decade ad much effort pet o the ubect givig a detailed review of motio etimatio techique would be hard ad impractical. I thi chapter mot popular method will be itroduced which how a cloer relatio with the work ivetigated here. Thee method ca be claified ito three mai categorie which are gradiet-baed block-matchig ad frequecy domai techique. Thi work ca be claified motly i the lat group. There are alo other techique uch a pel-recurive ad Bayeia algorithm. But motly three tadard motio etimatio techique will be emphaized ice they how a cloer relatio to the work ivetigated here. However a brief itroductio about pel-recurive ad Bayeia method i give. There are practical limitatio i the above-metioed method which ca be overcome by ue of multireolutio cocept or hierarchical approach. Thi ubect will be itroduced i chapter 3.. Motio Etimatio Techique Almot all motio etimatio techique are baed o the aumptio that ay chage i the image iteity i oly due to the motio i the cee. Thi mea that while a obect i movig alog a directio the image iteity of ay movig part of the obect remai cotat. Sice image iteity i a fuctio of time iteity at ay image poit = x x I x t. Uig cotat iteity T x at time itat t ca be repreeted a aumptio it ca be aid that ay poit i the image will appear with a diplaced locatio i a later time. 7

21 I +. x dx t + dt = I x t If the left ide of the equatio i rewritte uig a Taylor expaio I x + dx t + dt = I x t + I x t. I dx + t x t dt + εt. where i the gradiet operator with repect to the poitio parameter x ad ε T i the error term repreetig higher order term i Taylor erie expaio. Neglectig higher order term ubtitutig. i. ad dividig both ide by dt reult i a popular equatio kow a the optical flow cotrait OFC: I I x t v + = t x t..3 where the vector v = v v T dx = dt dx dt T repreet the velocity alog the motio directio. The olutio of the equatio reult i a traight lie i v pace a how i Figure.. Notice that preece of two variable for oe equatio make optical flow etimatio a ill-poed problem. OFC alo implie that at ay poit oly the compoet of the motio parallel to the image patial gradiet i.e. motio perpedicular to the image edge i recoverable. Thi i kow a the aperture problem which mea that the compoet of the motio alog the directio of a edge caot be determied ule the ize of the aperture of the aalyzig widow i larger tha the legth of the edge. A illutratio of the aperture problem i give i Figure.. From a dicrete poit of view the image of the 3 cee take at equally paced time itat ca be coidered a a equece of picture. By thi the time variable of the iteity fuctio of ay poit i the image ca be dropped ad a dicrete verio of equatio. ca be writte a v I k + + d = I k.4 8

22 V OFC cotrait lie V a x t I V b V c V Figure.. OFC cotrait lie with poible olutio v a vb ad v c Curret Frame Figure. The aperture problem Key: Obect i curret frame Obect i previou frame Regio uder coideratio Correct motio vector Poible motio vector where I k repreet the iteity value of a pixel at locatio = T i frame k ad d i the correpodig diplacemet vector. However ice thi diplacemet i the quatity to earch the equatio may be rewritte a a error fuctio of the diplacemet vector i.e. a imilarity meaure ca be writte betwee pixel i adacet frame a a fuctio of etimated diplacemet vector ε.5 F d I + d I = k + k 9

23 where ε i kow a the diplaced frame differece F. Whe F d i truly etimated the F will be zero uder cotat iteity flow ad tralatioal motio coditio. Therefore the diplacemet vector etimatio ca be doe by miimizig ε F with repect to d. Notice that equatio.4 hould fail i the cae of motio due to rotatio dilatio or occluio i the image. However if thee deformatio are mall the correpodig motio may be approximated by tralatio ad equatio.4 may become a reaoable aumptio. I uch a cae we may write I k + + d uig a Taylor erie expaio I d = I + I. d + ε.6 k + + k + k + which ca be ubtituted i equatio.5 yieldig ε.7 F d = I I + I. d k + k k + igorig higher order term. If we divide the above expreio by the time iterval t betwee frame ad take the limit a t we obtai agai the OFC i time domai. The motio etimatio techique that will be decribed ext ue oe of thee viewpoit i.e. they either ue the OFC directly by meaurig the patiotemporal rate of chage of the image iteity or they miimize the F over a et of local regio by earchig over a et of motio vector... Gradiet Baed Techique Gradiet-baed techique provide a etimate of the optical flow field i term of patiotemporal iteity gradiet. Uually the OFC i ued i couctio with a appropriate patiotemporal moothe cotrait which require that the velocity vector varie lowly over a eighborhood. Optical flow cotrait wa firt ued to etimate motio by Cafforio ad Rocca []. They attempted to egmet the image ito a fixed backgroud ad a movig obect regio uig dyamic programmig. They aumed velocity to be cotat withi the regio of movemet ad miimized the quatity

24 I x t ε OFC = I x t. v +.8 t Hor ad Schuck [] impoed a global moothe cotrait o the optical flow field from which a ecod quatity to be miimized come. Thi quatity i the um of quare of the patial gradiet of the velocity compoet ε = v.9 SC x t + v x t Combiig thee two quatitie reult i a total error to be miimized ε α ε global v = OFC + ε SC dx. image where α i the relative weightig factor betwee the two error term ad it idicate the importace of global moothe cotrait relative to the OFC. Equatio.8 ca be aalytically olved. But olutio of equatio. require a erie of Gau-Seidel iteratio for covergece to the miimizig olutio v x over the whole image. Itead of a global moothe cotrait Luca ad Kaade [3] propoed a local moothe cotrait by aumig that the motio vector i the ame for a particular image regio R. Thi ca be expreed a the miimizatio of the quared F with repect to the diplacemet vector d ε d = w x local xεr ε F. where w x i a widow fuctio givig more weight to the cetral part of the regio. Uig equatio.7 ad igorig the higher order term the miimizatio of. with repect to the diplacemet vector d ca be expreed a xεr w x [ I x I x + I x d x ] I x k + k k +. k + =. which repreet a overdetermied ytem of liear equatio o the two diplacemet compoet ad it olutio may be foud uig tadard leat quare techique.

25 A alterative to the global ad local moothe cotrait i to ue a cotrait o ecod order derivative of the image iteity fuctio for the velocity vector. The I x t i.e. cotrait i the coervatio of the patial iteity gradiet d [ I x t ] dt =.3 which give I x I x x I x x I x v v I t x + = I t x.4 But for thi method to be ucceful there mut ot be firt-order deformatio of iteity i the image uch a dilatio or rotatio which i commoly preet i real image equece. The problem with thi aumptio i that it doe ot allow for firt-order deformatio of iteity uch a dilatio or rotatio commoly preet i image equece. Becaue of thi ecod-order method ofte produce le accurate etimate tha other method. Both local ad global gradiet-baed method rely o the aumptio of moothe of the optical flow field. The local method preet difficultie i regio where the patial gradiet chage lowly. The local iformatio i iufficiet ad therefore cotrait equatio ariig from differet eighborig poit provide eetially the ame cotrait o the optical flow. The global method reduce thee problem ice the iformatio i propagated over the image. However the error are alo propagated. I thi cae the mai problem i that the moothe cotrait doe ot hold acro motio boudarie i.e. it blur motio edge. Gradiet method alo uffer from the fact that the OFC i ofte violated. Sice the Taylor expaio ued i order to derive thi cotrait i trucated i.e. higher order term are eglected the OFC break dow for large motio vector. A ubcla of gradiet-baed method kow a pel-recurive iterative gradiet algorithm repreet a attempt to i overcome thi problem. I thi algorithm a iitial diplacemet field d i computed. i Thi i the updated recurively uig the Taylor expaio of I x + d aroud i d

26 producig a refied etimate d i+. The mai drawback of thee algorithm i that the iterative procedure may coverge to a local miimum rather tha to the global oe. Bayeia techique [4] which may be coidered a a geeralizatio of gradiet-baed techique make ue of probabilitic moothe cotrait i additio to the OFC i order to etimate the motio field. Sice optical flow field iclude image oie lightig chage low cotrat regio aperture problem ad multiple motio i a igle localized regio a probabilitic framework would allow thee ucertaitie to be repreeted i the computatio ad paed alog to the ext tage of computatio. For thi purpoe a coditioal probability fuctio of the velocity baed o the image iteity gradiet may be ued. Other techique developed to deal with larger motio are hierarchical gradiet-baed method. Thee are uually baed o a Gauia pyramid which i a multireolutio tructure compoed of ucceively moother ad ubampled verio of the image. Thee image verio are obtaied through iterative filterig of the image with a Gauia kerel followed by ubamplig. The motio etimatio i performed i a coare-to-fie way i.e. etimatio tart at the coaret reolutio ad thee etimate are ued a iitial poit to motio etimatio at the ext fier reolutio... Block-Matchig Techique Block-matchig techique ca be coidered a the firt ad mot popular method for practical motio etimatio becaue of their le hardware complexity requiremet ad implemetatio eae. I block-matchig techique a regioal direct earch procedure o image i ued itead of uig a Taylor erie expaio to etimate d i term of the iteity derivative. The tartig poit of block-matchig techique i the F exteded over a regio amed a diplaced regio differece R R m xεr I x + d I x R d = z.5 m+ m where z i a ditortio fuctio ad m repreet the umber of picture i frame equece. The mot commo ditortio fuctio are mea quare error MSE ad maximum abolute differece MA. The aim of block-matchig method i to miimize the R ad fid the motio etimate d for which the R i miimum i.e. 3

27 = arg mi { R R d } dˆ R.6 d m Block-matchig algorithm work by partitioig the image ito block of the ame ize ad the aigig the ame motio vector to all the pixel icluded by the block. For each block B the motio vector i evaluated by matchig the iformatio cotet of the block i frame k with that of block of the ame ize withi a earch area S placed i the followig frame k+ a how i Figure.3. I order to fid a abolute miimum for the matchig criterio.5 a exhautive earch of a erie of dicrete cadidate diplacemet withi a maximum diplacemet rage mut be performed. Thi techique i called full-earch block matchig. epite the heavy computatio it require it i widely ued i video codig due to it implicity ad eae of hardware implemetatio. Itead of miimizig the quared F a imilar approach i to maximize the crocorrelatio betwee the block i the two frame. From the defiitio of the F.5 ad equatio.6 the expreio to be miimized for geerally ued ditortio fuctio MSE i [ I d + I I d I ] k + + k k + + k.7 εb From thi expreio it ca be ee that fidig the diplacemet d that miimize.7 i equivalet to fidig d that maximize frame k+ frame k Block earch widow Figure.3 Block-matchig earch algorithm 4

28 C B d I d [ k + I k ].8 = + εb which i the expreio for the cro-correlatio defied over a block B of the iteity fuctio of frame k ad k+. A problem with block-matchig techique i the high computatioal expee. For example for a block cotaiig N pel a full earch over a regio R compriig r cadidate require 3 N r additio ad multiplicatio operatio. However fat earch algorithm have bee propoed i order to decreae the computatioal expee of the fullearch algorithm. But they decreae the accuracy of the method. The mot popular earch algorithm are 3-tep earch - earch cougate directio earch ad hierarchical earch. Aother problem with block-matchig techique i that they produce iteger etimate ice the earch i performed over a dicrete grid. I order to produce ub-pixel accuracy the image iteity ha to be iterpolated at fractioal pixel locatio. The block ize i oe of the mot importat parameter i ay block-baed motio etimatio algorithm. Selectio of the block ize eed a optimizatio becaue of two fact. The widow mut be large eough i order to be able to etimate large diplacemet vector Otherwie umber of fale matche with large diplacemet vector icreae. O the other had it hould be mall eough o that the diplacemet vector remai cotat withi the widow. Thee two requiremet ca uually be addreed by hierarchical method. For itace Aada propoed a block-matchig coare-to-fie algorithm baed o a Laplacia pyramid [5] which i a multireolutio tructure uch a the Gauia pyramid but the pyramid image repreet badpa verio of the origial oe rather tha lowpa. Itead ufaux ad Mochei [6] ued a adaptive multigrid tructure i which the image i iitially divided ito a few large block ad a motio vector i etimated for each block. The the correpodig block i equally divided ito four childre block which iherit a motio etimate which i a combiatio of the cloet four paret block. Thi etimate i the refied uig a block-matchig techique at that grid ad the proce i repeated util the fiet grid...3 Frequecy omai Techique Frequecy domai techique etimate motio by traformig the image equece to the frequecy domai ad the miimizig ome diparity meaure or maximizig a correlatio meaure baed o the traform coefficiet rather tha image iteitie [9]. Approache have bee developed where the relevat meaure i a fuctio of either the 5

29 magitude or the phae of the traform coefficiet give rie repectively to eergy-baed ad phae-baed techique. The cro-correlatio expreio give by equatio.8 ca be computed more efficietly i the frequecy domai by uig the property of the Fourier traform that covolutio i the pace domai i equivalet to multiplicatio i the frequecy domai. I B k Ω where d F { I Ω Iˆ Ω } ˆ.9 C B = B k + B k ˆ i the Fourier traform of I k over the block B i frame k w T Ω = i the patial-frequecy vector ad w F idicate the ivere Fourier traform. Therefore the diplacemet vector d may be etimated by locatig the peak of the cro-correlatio fuctio give by.9. Thi method i equivalet to performig a local cro-correlatio where the widow fuctio i a rectagular widow. Other approache ue phae correlatio rather tha the magitude correlatio to etimate motio. The idea of phae correlatio-baed motio etimatio i origiated from the hiftig property of the Fourier traform. f f t F w iwτ t + τ e F w. If equatio.4 i writte i frequecy domai uig equatio. we get Iˆ. k + Ω = Ω Ω T i d e Iˆ k where the diplacemet ca be obtaied by dividig the Fourier phae differece by agular frequecy ω. However the Fourier traform decribe behavior over all time ad hift which are localized to a particular time iterval ca ot be etimated by thi method. To etimate local hift we eed a decriptio which i repoive to local chage i the igal. For thi purpoe Short-Time Fourier Traform STFT i propoed which widow the Fourier bai fuctio to provide a time-frequecy decriptio of the igal STFT τ + T wt x τ w = x t w t τ e dt. τ T 6

30 where w t i a real-valued ymmetric fuctio defied over a fiite iterval [ TT ] i.e. a widow. Thi traform ca be coidered a a aalyi of a igal by filter of fiite extet i time ad frequecy. The product of the time ad frequecy reolutio deped o the form of the widow w. The Gabor traform for example ue a Gauia widow which achieve the bet poible combiatio of frequecy ad time reolutio [8]. Fleet ad Jepo [7] ued the phae output of everal Gabor filter tued to differet patiotemporal frequecie i order to etimate the motio. They aumed that the phae of the filter output i cotat alog motio traectorie edig up with φ x t φ x t. v + =.3 t i.e. a optical flow cotrait o the phae φ of each filter output rather tha o iteity. They ued thi equatio to etimate ormal velocitie i.e. alog the phae patial gradiet φ xt ad combied the etimate obtaied from differet filter over local regio to produce two-dimeioal motio etimate. To upport their ue of the phae rather tha of the amplitude of the Gabor filter they made imulatio to how that the evolutio of cotat phae cotour wa cloer to the - proectio of the 3- motio of the cee tha the evolutio of cotat amplitude cotour. Alo they howed that phae iformatio i robut with repect to mooth illumiatio chage ad mall deformatio caued by the perpective proectio of movig three-dimeioal obect which caue deviatio from the imple tralatio motio model ued i mot optical flow techique. The CWT motio etimatio algorithm ivetigated here i alo related with the frequecy domai cla of motio techique. A i Fleet ad Jepo method motio i etimated from the phae output of Gabor-like filter. However thee filter are implemeted efficietly through a cheme derived from the dicrete wavelet traform of Mallat []. The method i decribed with more detail i the ext chapter.. Summary Thi chapter ha reviewed the mot widely ued trategie to the problem of motio etimatio i particular the oe belogig to the clae of gradiet-baed block-matchig ad frequecy domai techique. 7

31 Gradiet-baed method produce real-valued motio field but have limited meauremet rage ad are uceptible to oie becaue of their ue of patiotemporal derivative. Blockbaed matchig method have arbitrary rage ad good immuity to oie but are limited i preciio ad are computatioally expeive. Frequecy domai method have i priciple ifiite preciio but greater computatioal cot tha gradiet-baed method becaue of the preproceig tage for traformatio. I particular phae-baed method have demotrated robute to the failure of the tralatig-regio aumptio i real equece. However they ofte lack robute to oie. Hierarchical or coare-to-fie etimatio i a computatioally efficiet mea of icreaig meauremet rage ad alo improvig oie immuity. Ay etimatio algorithm may be implemeted i a hierarchical framework uig a multireolutio repreetatio of the iput image uch a a Gauia or Laplacia pyramid. I the ext chapter a hierarchical phae-baed motio etimatio algorithm baed o complex dicrete wavelet traform i itroduced which combie efficiecy preciio ad robute. 8

32 CHAPTER 3 MOTION ESTIMATION BASE ON A COMPLEX ISCRETE WAVELET TRANSFORM The wavelet traform ha become a popular techique epecially i the pat two decade which foud applicatio i variou field uch a igal ad image proceig patter recogitio ad computer viio atroomy acoutic ad geophyic. There are two mai factor for thi popularity. The firt reao i due to the itriic ature of the wavelet traform which ha a good time-frequecy localizatio property. I order to udertad thi tadard Fourier techique mut be aalyzed firt. Thee techique aalyze the total frequecy cotet of a igal uig ifiite expoetial wave. Sice the Fourier traform FT i uited to the aalyi of periodic igal if the frequecy cotet of a igal at a particular time i required e.g. if the itat at which a pike occurred i the igal i required the FT will be uable to provide the iformatio. The lack of FT i that it provide good frequecy localizatio but o time localizatio. To overcome thi problem the hort-time Fourier traform STFT wa developed which aalyze the igal at pecific time locatio by uig widowed expoetial wave. Thi i doe through multiplicatio of the expoetial wave by a uitable widow fuctio cetered at the required time itat. The Gabor traform ued i thi work i a particular cae of the STFT which ue a Gauia kerel a the widow fuctio. Gabor [8] proved that with thi widow the STFT achieved the bet oit time-frequecy localizatio. Beide the good oit time-frequecy localizatio property STFT till preet a problem of cale depedecy caued by the widow fuctio. To udertad thi thik agai of eekig the time locatio of a pike i the igal. If the width of the pike i much maller or larger tha the width of the aalyi widow there will be till a problem of uability to pecify the time itat of pike occurrece. To overcome thi problem WT which aalyze 9

33 a igal at differet time locatio with kerel of varyig ize i ued where the kerel are formed by tralatio ad dilatio of a prototype fuctio called the mother wavelet. The ecod factor for the popularity of the wavelet traform i due to Mallat []. He itroduced the idea of multireolutio decompoitio ad howed that the wavelet coefficiet were uitable to repreet the iformatio differece betwee two differet reolutio verio of the ame igal. He alo poited out the imilarity betwee the wavelet traform ad quadrature mirror filter QMF bak. Thi eable for a efficiet implemetatio of a dicrete verio of the WT WT. Combiig the cale idepedece ad time-frequecy localizatio propertie of the wavelet traform with the relatio of local tralatio i the image with FT phae give a good idea of a approach to ME problem. I thi chapter we decribe the complex dicrete wavelet traform baed ME approach of Magarey ad Kigbury [ 9]. Thi traform combie the efficiecy of a QMF implemetatio with the fact that the correpodig filter may be modeled a caled Gabor fuctio providig i thi way the deired Fourier/wavelet combiatio. 3. THE COMPLEX ISCRETE WAVELET TRANSFORM I thi ectio we briefly review the cotiuou ad dicrete wavelet traform i oe ad two dimeio a a bai to the more detailed decriptio of the complex dicrete wavelet traform of Magarey ad Kigbury [ 9]. 3.. ONE-IMENSIONAL WAVELET TRANSFORM 3... Cotiuou time The cotiuou wavelet traform of a oe-dimeioal igal I x i defied a [ ] { I x } α = WT = τ I x ψ x dx 3. τα where ψ x repreet a et of wavelet geerated from the mother wavelet ψ x by a τα chage of cale the cale of ψ x i covetioally ad i ψ x the cale parameter α ad a tralatio i time the fuctio ψ x i covetioally cetered aroud ad τα

34 ψ τα x i the cetered aroud τ. The et of wavelet geerated from the mother wavelet i formulated a follow. ψ τα x τ x = ψ 3. α α where α τεr. Notice for the cale factor α that very large cale mea global view tretched wavelet while very mall cale mea detailed viewhruk wavelet. It i obviou that the cotiuou wavelet traform i highly redudat for repreetatio of a igal. Thi redudacy may be reduced by amplig the cale ad tralatio parameter reultig i the dicrete wavelet traform. aubechie [] ued a dyadic amplig for α ad a liear proportioal to the cale amplig for τ i.e. α = α ad τ = kα kεz. The traform ca the be writte a a et of iteger-idexed coefficiet where k k = ψ x I x dx 3.3 x k ψ x = ψ 3.4 k 3... icrete Time I mot igal proceig applicatio the igal i ampled i time I I εz a dicrete verio of the cotiuou wavelet traform WT ca be write a = ad or k = I k ψ 3.5 k = I =... ψ 3.6 where ψ are dicrete-time wavelet filter ad the operator repreet dowamplig of time. The mai differece betwee thi traform ad it equivalet i cotiuou time i that the filter ψ are ot perfectly caled verio of oe aother[] i.e. they do ot atify the relatio

35 x = ψ x= ψ 3.7 However thi relatio ted to hold a Wavelet Traform A a Filter Bak I [] Mallat itroduced the multireolutio aalyi cocept ad howed the uitability of wavelet bae to repreet the differece i iformatio betwee approximatio of a igal at differet reolutio. I tur there i a cloe coectio betwee a multireolutio repreetatio ad a ubbad filterig cheme thu makig it poible to implemet the WT defied by 3.5 uig thi cheme. I a ubbad filterig cheme the iput igal I i plit ito two ad imultaeouly aalyzed by a pair of halfbad filter a lowpa filter h ad a highpa filter h followed by dowamplig by a factor of a how i Figure 3.. The highpa filter provide the firt level of detail after dowamplig by while the dowampled lowpa output become a coare approximatio to the iput igal each of which have half the reolutio but double the cale. The output of the lowpa brach i the iput to the ext tage ad aalyzed i the ame way a the origial igal. Thi tructure i repeated a required reultig i a ubbad decompoitio tree how i Figure 3.. Note that due to the dowamplig operatio the umber of output ample equal the umber of iput oe therefore the output igal i critically ampled i.e. ha o redudacy. The relatio betwee the WT ad the ubbad filter bak i that the output of the highpa brache may be iterpreted a the reult of the covolutio of the igal I with the correpodig dicrete wavelet filter ψ of equatio 3.5. Sice the filter pair Figure 3. Two-bad buildig block for dyadic WT.

36 Figure 3. Subbad decompoitio tree 3 level { h h } bear a direct relatio with the wavelet filter decriptio of digital filter thi relatio may be writte a Ψ k z H z k = ψ uig the Z-traform z = H 3.8 where H HadΨ tad repectively for the Fourier traform of h hadψ ad iw z = e. Equatio 3.8 how that i order to obtai the th level coefficiet of the traform the iput igal eed to be lowpaed ad dowampled - time ad ubequetly highpaed ad dowampled oce. The correpodig reidual of thi treei.e. the output of the th lowpa filter may be obtaied by a imilar expreio to 3.5 or k I = I k φ 3.9 k I I = φ 3. where φ i the equivalet calig filter defied by the expreio 3

37 Φ k z = z H 3. k = ote that Φ i the Fourier traform of φ. A the average value or C compoet of the iput igal. filterig by φ ted to give 3.. Two-imeioal icrete Wavelet Traform A Mallat [] how the wavelet traform i two dimeio may be implemeted mot efficietly by uig a two-dimeioal eparable filter. Thi correpod to writig two-dimeioal mother wavelet a a multiplicatio of two oe-dimeioal fuctio ad proceig the colum ad the row of the image I a i the tructure how i Figure 3.3. The reult i four ubimage each havig oe quarter ize of the origial image o the eparable -d WT ha o redudacy a i the -d cae. I i a coare verio of the iput image I. It ha bee lowpa filtered by h ad dowampled i each directio ad therefore repreet a half-reolutio ad double-cale verio of the origial image. The I i ued a the iput to the ext level of the ubbad tree. The remaiig badpa ubimage outcomig from the applicatio of h h h h ad h h followed by correpodig dowamplig i both directio correpod to the twodimeioal wavelet coefficiet where cotai detail i 3 the { } horizotal vertical ad diagoal directio repectively. Mathematically the lowpa ubimage coefficiet at cale are Figure 3.3 Buildig block for eparable WT o image I. 4

38 I k = I k φ 3. k ad the badpa ubimage coefficiet at cale ad ubbad are k = I k ψ 3.3 k where φ ad { ψ = 3 } are the correpodig two-dimeioal calig ad wavelet filter give by φ = φ φ ψ =ψ φ ψ = φ ψ 3 ψ =ψ ψ Figure 3.4 how the partitio of the uit ormalied frequecy cell by a igle-level WT. Note that at a give cale each of the above defied wavelet filter emphaie feature i the iput image poitioed at differet orietatio. Therefore the filter ψ which i a Figure 3.4 Partitio of the two-dimeioal uit frequecy cell by a igle-level eparable WT 5

39 combiatio of a highpa filter i the vertical directio ad a lowpa filter i the horizotal directio emphaie horizotal edge while the filter ψ which i the oppoite 3 combiatio emphaie vertical edge. Fially the filter ψ emphaie diagoal edge but ote that it doe ot make ditictio betwee diagoal edge orieted at π 4 or 3π 4 radia. A olutio to the ditictio problem of diagoal edge will be itroduced i ectio through the itroductio of mirror filter. To ummarie the eparable -d WT decompoe a image ito a hierarchy of ubimage of differet cale ad orietatio by uig a et of ditictly caled ad orieted patial filter COMPLEX WT I thi ectio the complex dicrete wavelet traform CWT developed by Magarey ad Kigbury [ 9] will be itroduced. Thi traform ue a complex-valued filter pair { h } h itead of the real-valued pair ued i Mallat WT ubbad tree decribed before. The CWT wa developed to be ued i the phae-baed motio etimator decribed before Oe-imeioal CWT Up to ow we implemeted WT with real-valued filter. Sice the ME algorithm ue the phae of a complex-valued traform the { h h } pair mut be complex-valued ad they may be modeled a Gabor filter [ 9] h a e σ e iw 3.8 h a e σ for =-L...L- e iw 3.9 where L i the legth of the filter i et to to poitio the Gauia widow ymmetrically i the iterval [-L L-] ad w ad w are the ceter frequecie. For proper choice of σ ad σ Fourier traform of equatio 3.8 ad 3.9 ca be write a 6

40 H H w = a w = a for w ε[ π ] w w σ iw σ π e e 3. w w σ iw σ π e e 3. For a particular choice of the parameter a a σ σ w ad w ectio 3..3 the equivalet wavelet ad calig filter equatio 3.8 ad 3. may alo be approximated a Gabor fuctio [ 9] σ iw ψ a e e 3. ˆ σ iwˆ aˆ e e φ 3.3 for =- -L... - L- with =. The parameter a aˆ σ ˆ σ w ad ŵ ca be calculated from the parameter of h ad h uig equatio 3.8 ad 3.. The wavelet ad calig filter give by 3. ad 3.3 have the ame characteritic a the real-valued oe i term of ot beig perfectly caled verio of oe aother. However a perfect calig i a deirable property for the applicatio i mid a prefilter f i applied to the iput igal before the firt level of the tree. Thi prefilter i defied by the equatio h * f λ. f = 3.4 where λ = H ad H i the Fourier traform of h. The purpoe of the prefilter f i to imulate a ifiitely large WT tree. Coequetly the reultig f-modified equivalet wavelet ad calig filter atify the relatio 3.7. The perfectly caled oe-dimeioal CWT ca be implemeted a a tadard WT Figure 3. except that the filter for the firt level are h f = h * f ad h f = h * f itead of h ad h. Notice that i a perfectly caled -d wavelet decompoitio with it prefilter f ha equivalet wavelet ad calig filter f-modified a follow 7

41 f = ψ f φ f ψ 3.5 φ = 3.6 f * m m I tur the f-modified wavelet ad calig filter ψ ad φ ca till be approximated a f f Gabor filter: 3.7 ψ f a mf e σ mf mf e iwmf mf mf ˆ σ mf iwˆ mf mf φ aˆ e e 3.8 f mf where agai the parameter ca be calculated from thoe of h ad h where w = w 3.9 f f w ˆ = w 3.3 f f howig that the ratio w wˆ i idepedet of m a deired. f f Note: From ow o we aume the preece of the perfect-calig prefilter f ad therefore drop all f ubcript o filter ame ad parameter Two-imeioal CWT I cae of two dimeio the traform i implemeted a previouly decribed i -d for real-valued filter i.e. i a eparable way proceig firt the colum ad the the row of the iput image. A previouly metioed the reultig two-dimeioal wavelet ad calig filter may alo be modeled a two-dimeioal Gabor fuctio which are give by the multiplicatio of the correpodig oe-dimeioal filter a i equatio 3.4 to 3.7. I practie filterig a image with thi et of two-dimeioal Gabor fuctio reult i a orietatio aalyi i.e. each filter emphaie feature withi the iput image which are orieted i the preferred directio of the filter. Thee directio are perpedicular to the correpodig two-dimeioal ceter frequecy vector. 8

42 Figure 3.5 Elliptical quai-circular cotour of the magitude repoe of the twodimeioal CWT wavelet filter at level ad -. Ω = w wˆ T ŵ w T Ω = 3.3 Ω 3 = w w T Mirror filter The -d CWT wavelet filter have igificat magitude repoe oly i the rage [ π ]. Sice the -d wavelet filter are product of the -d wavelet ad calig filter they alo have igificat magitude repoe i the ame rage. Thi implie that the output of the correpodig two-dimeioal wavelet filter i maily localied i the firt quadrat of the uit frequecy cell a how i Figure 3.5. I term of the orietatio electivity ut metioed thi i equivalet to aalyig oly orietatio ragig from to π. However real-valued image cotai igificat iformatio i the firt ad ecod quadrat of the uit frequecy cell the third ad fourth quadrat are cougated verio of the firt ad ecod quadrat. I order ot to looe ay iformatio whe movig to the CWT domai the ecod quadrat egative horizotal frequecy poitive vertical frequecy of the uit frequecy cell i.e. orietatio withi the iterval [ π π mut be covered. For thi purpoe a et of mirror filter i added to the filter bak. Thi i achieved by uig the complex cougate 9

43 * h ad * h i parallel with h ad h whe filterig the row of the image Figure 3.6. Cougatig h ad h reflect their magitude frequecy repoe about w= o the cougated filter cover the frequecy rage [ π ] Coequetly the output of the firt level of the CWT tree i eight complex ubimage two beig lowpa verio of the origial image ad ix correpodig to the two-dimeioal wavelet coefficiet. The ubimage comig from the cougate filter brach cotai iformatio related to the ecod quadrat of the uit frequecy cell ee Figure 3.7. I thi way the CWT ditiguihe betwee orthogoal diagoal edge which i ot true for the tadard realvalued WT. Figure 3.6 Two-dimeioal CWT level Figure 3.7 The magitude repoe of the mirror filter coverig the ecod quadrat of the uit frequecy cell. 3

44 The ubequet level of the tree are computed by applyig h ad h a uual to the firt quadrat lowpa ubimage ad the mirror filter to the ecod quadrat lowpa ubimage. Overall thi implie a redudacy of 4 to for the CWT idepedet of the depth of the tree. Hece the complex verio of the WT i ot critically ampled however thi redudacy i crucial for the ucce of the motio etimatio method Sectio 3.. The two-dimeioal mirror calig ad wavelet filter are * φ = φ φ 4 * ψ =ψ φ 5 * ψ = φ ψ 6 * ψ =ψ ψ ad the correpodig ceter frequecie of the mirror wavelet filter are give by Ω Ω 4 5 Ω 6 = = = w wˆ T wˆ w T w w T Cotributig Regio At each cale of the traform a pixel withi a ubbad or a ubpel cotai iformatio comig from a eighborhood urroudig the origial pixel. The ize of the cotributig eighborhood at each level i relatio to the pixel deity of the origial image may be etimated by coiderig a approximately circular regio with radiu correpodig to twice the variace σ ee equatio Summary I thi ectio -d CWT which i built eparably by uig complex-valued filter modeled a Gabor Kerel i itroduced. Thi traform aalye a image by decompoig it ito a et of ix orietatio-tued ubbad at differet reolutio. The icluio of the mirror filter prevet iformatio lo o traform by icludig the ecod quadrat of the uit frequecy cell ad provide emphaizig of diagoal orietatio with differet ubbad. Figure 3.8 how a example rutarget image ad it CWT 3

45 decompoitio 5 level. The upper right quadrat how the real ad imagiary part of the fiet vertical ubbad top left ad right of thi quadrat plu the real ad imagiary part of the mirror ubbad bottom left ad right. I the ame way the lower right quadrat how the diagoal ubbad ad the lower left quadrat how the horizotal ubbad. The upper left quadrat how the ame tructure for the ext coarer reolutio a o o. The mallet image i the upper left corer correpod to the lowpa reidual. Notice that rutarget picture i ued becaue of it wide-pectrum. 3. Motio Etimatio Algorithm Figure 3.9 how the hierarchical tructure of our algorithm. I ad I are the referece ad curret frame repectively which are traformed by CWT decompoitio from top to bottom producig ix complex badpa ubimage at each tage ad the iput to the algorithm. The CWT badpa ubimage at each level are iput to the motio etimator Figure 3.8 Frequecy repoe of ubbad filter i -d CWT decompoitio. 3

46 Figure 3.9 Hierarchical tructure of CWT-baed motio etimatio algorithm 33

47 wherea the lowpa ubimage are iput to the ext decompoitio tage. After CWT decompoitio i completed motio etimatio tart by matchig the traform coefficiet at the coaret level max producig a motio vector for each ubpel at thi reolutio. The algorithm work the from coare to fie uig at each level the correpodig traform coefficiet plu the etimate of the previou coarer level to produce a deer ad more accurate motio field. Our algorithm produce a igle motio etimate at each ubpel at a give level. Becaue we ue a dyadic WT each level- ubpel directly ubted four level - ubpel. The output field of each etimator mut be quadrupled i ize iterpolated by i each directio ad caled up by before it ca be ued a a iput to the ext tage. The motio etimate are ued a tartig poit for the ext fier level of etimatio by mea of a warpig or ubbad iterpolatio tep. Thi proce i repeated util a pre-et fiet reolutio mi i attaied. The algorithm halt at level mi where the motio field ha a deity of mi mi i.e. the field cotai oe motio etimate per block of mi by pel mi. To obtai a full reolutio motio field i.e. oe motio vector for every pixel i the origial image the motio field at level mi i upampled ad iterpolated mi time. 3.. SINGLE LEVEL ESTIMATION 3... Subbad Squared ifferece The algorithm i baed o a matchig criterio at ubpel ad ubbad give by S f = f 3.37 where f i the offet vector. Sice at leat two ditict orietatioal ubbad are eeded to obtai uique etimate ad all ix orietatioal ubbad to obtai a robut etimate ummig ix ubbad with a weightig factor give the reultat matchig criterio called the ubbad quared differecess give by S 6 f 3.38 ε P = f = 34

48 where ad are the CWT coefficiet at level ad ubbad of the referece ad curret frame repectively. f i the diplacemet of ubpel at cale which f at the origial reolutio of the iput image. P amout to a diplacemet of i a weightig factor correpodig to the eergy of the wavelet filter at ubbad ad cale give by P = π ππ Ψ Ω dω 3.39 where Ψ i the Fourier traform of ψ. The idea behid thi factor i to elimiate the filter depedecy whe combiig the differet ubbad. I tur ε i a factor itroduced to avoid the cale depedecy of P whe combiig the SS over differet cale[]. The SS i aalogou to the quared F i the iteity domai which wa decribed i chapter. But there are two igificat differece betwee thoe quatitie. The firt oe i i the cae of the SS the average i performed over the et of orieted ubbad with the filter eergy weightig factor rather tha over a et of eighborig pixel i the image a i F cae. The ecod oe i that the computed diplacemet f for the SS i real-valued rather tha iteger-valued CWT Coefficiet Iterpolatio ad Poitio Shiftability The reao why thi real-valued diplacemet vector may be computed follow from the iterpolability property of the CWT i.e. the ability of the CWT to produce real-idexed coefficiet from the kow iteger-idexed oe. I tur thi i poible due to aother property termed hiftability which CWT approximately atifie.[9]. Therefore we ca write f W k k 3.4 k f where W i a lowpa kerel modulated to the ceter frequecy of the equivalet f wavelet filter i.e. 35

49 W ϖ f T i Ω k f k = H k e 3.4 f where H f i a eparable lowpa kerel give a H k h f k h f f = k 3.4 The -d iterpolator fuctio h f k ca be either Widowed Sic or the Lagrage Iterpolator [9]. Notice that the rage of k deped o the rage of f value required. I practie oly a uit rage i each directio eed to be coidered becaue the CWT coefficiet may alway be iteger-hifted to cope with value that fall outide that rage. For example f f f f + = + + = + f where = + f ε Ζ f = f f ε ad [ ] 3.43 Here the ymbol f idicate the floor operator o f i.e. the iteger immediately to the left of f o the umber lie. The implet choice for the oe-dimeioal lowpa kerel h f k i the taircae iterpolator which i the trivial Lagrage iterpolator L= if k = h f k = otherwie 3.44 for f ε [-.5.5] 3.45 I thi cae equatio 3.4 become T i Ω f f e 3.46 for ε[.5.5] [.5.5] f

50 37 The above expreio aume that for mall f hift the magitude of the CWT coefficiet i approximately cotat with oly the phae chagig accordig to the cetre frequecy of the correpodig wavelet filter. I hi thei [9] Magarey report that thi model hold up well over the uit iterval provided the origial image ha o trog pectral compoet i the pabad of the filter but i reaoably pectrally flat. Therefore a the filter badwidth decreae a icreae thi property i more readily atified at the lower reolutio where the greatet accuracy i required Quadratic Surface Coider the ubbad quared differece for oe particular ubbad f f S = 3.48 Thi equatio may be further expaded ito { } * f e f f S R + = 3.49 ad the SS ca be further approximated a f f S.co θ θ + for [ ] [ ] ε f 3.5 where f θ ad θ are the phae of f ad repectively. Thi how that miimizig S i very likely maximiig the phae correlatio betwee two ubpel. We ca characterie the S urface aroud it miimum lie by uig the approximatio co x x ad the taircae model provided by equatio 3.46 we fid f f S T θ Ω + = 3.5

51 3 3 Figure 3. Cotour of urface S =...6 ad their um S cetre bottom for a typical diplacemet etimatio. where θ i the phae differece betwee ad defied by equatio 3.5 θ = = θ θ 3.5 Expreio 3.5 repreet a quadratic urface whoe miimum i a traight lie [] i.e. S i a quadratic urface with parabolic cro-ectio ad cotour parallel to it miimum lie a how i Figure 3.. If the miimum lie correpodig to each ubbad do ot lie too far apart the um equatio 3.5 i alo a quadratic urface ad it ca be writte i the form S f Af + Bf + Cf f + f + Ef + G 3.53 = or equivaletly S f = α f f + β f f + γ f f f f + δ

52 39 where f ad f are repectively the vertical ad horizotal compoet of the real offet f. The parameter A B C E G are fuctio of the coefficiet ad of the cetre frequecie Ω ad of the phae differece θ ad ca be directly derived from equatio give by the expreio = Ω = 6 A 3.55 = Ω = 6 B 3.56 = Ω Ω = 6. C 3.57 = Ω = 6. θ 3.58 = Ω = 6. E θ 3.59 = = + = 6 6 G θ 3.6 where T m Ω Ω = Ω 3.6 By uig the above derived parameter the urface miimum locatio i give by the expreio = = C AE CE B AB C f f f where the curvature parameter αβγ = A B C ad the urface miimum value i give by f Cf Bf Af G = δ 3.63

53 Figure 3. Miimum lie of quadratic urface give by expreio 3.5 a whe the motio i well-defied the miimum lie correpodig to the differet orieted ubbad cloely itercept each other b whe the motio i ot well-defied which idicate the cloee of the match betwee ad over =...6. f It i iteretig to ote propertie of thee urface. The parameter { α β γ } pecify the orietatio ad eccetricity of the ellipe. I geeral the larger they are the teeper the urface i at the miimum poit hece the cloer lie the itercept with each other of the miimum lie ariig from the idividual ubbad e.g. a i Figure 3.a Thi mea that the correpodig motio etimate miimum coordiate i more precie high cofidece. Iverely if the urface i flat i a particular directio mall curvature parameter it idicate ureliability low cofidece i the compoet of the motio i that directio e.g. a i Figure 3.b. Thi ha a cloer relatio with the aperture problem metioed i Chapter. The compoet of the motio alog the directio of a edge ca ot be determied ule the ize of the aperture of the aalyig device i larger tha the legth of the edge. We will refer agai to thee propertie whe dicuig curvature correctio Sectio There are alo ome pecial cae for fidig the miimum locatio. Equatio 3.6 will fail i fidig the miimum locatio where C = 4AB. There are two poible caue for thi: The ull cae A=. Sice A i o-egative ee Equatio 3.54 thi mut be the zero-activity cae i which all other parameter are zero a well. I uch cae et f. = 4

54 Figure 3. Choice of miimum locatio cloet to the origi i cae whe degeerate. S i The degeerate cae C 4AB =. Thi ituatio i the maifetatio of the aperture problem i the CWT domai. Note that each idividual o-ull urface S fall ito thi category o ay igal that excite oly oe of the ix ubbad will produce a degeerate S. I thi cae a uique miimum ca ot be foud. Itead there i a miimum lie pecified by Af + Cf = where f i et to the value o the lie with the mallet perpedicular ditace from the origi a how i Figure 3. give by the expreio A f = + C C A Beide thi pecial cae it i obviou that whe a compoet of f outide the rage [.5.5] i produced ull value are alo aiged. Becaue that rage i the abolute limit of validity of the taircae approximatio equatio 3.46 o which ME algorithm i baed. 4

55 3.. Hierarchical Etimatio 3... Iterpolatio ad Scalig of Quadratic Surface A previouly metioed the hierarchical approach to ME i to ue coare level etimate a tartig poit or iitial guee. Therefore the firt tep i to compute the coaret level max motio etimate which i output i the form of a field of S parameter { f α β γ δ } or { A B C E G} f max S =. I other word the parameter f α β γ δ are calculated for every ubpel at that level. Before the f max S i iput to the ext fier tage max of the motio etimatio it mut be iterpolated to attai the required pixel deity ad caled to accout for the chage i coordiate ytem f f producig a ew field S max. The iterpolatio i doe eparately i the colum ad the row by upamplig each columrow followed by filterig with a iterpolatio kerel. The implet iterpolatio i taircae i which each parameter i imply copied to it four fier level eighbour. Aother choice i the biliear kerel give by [ 33 ] 4. The calig factor are α A α / 4 A/ β B β / 4 B / γ C γ / 4 C / E / E / 3.69 f f f f Cumulative Squared ifferece Oce obtaied the field max S motio i etimated at level followig the max ame procedure i.e. uig equatio The reultig field of parameter max S i added to S max formig the cumulative quared differece CS at level max. Thi proce iterpolate fid ext etimate add i repeated iteratively i.e. CS CS f = + f + S S f f mi < = max max 3.7 4

56 util the deired reolutio mi i attaied. I thi way iformatio from all reolutio i icorporated ito oe igle quadratic urface to form the fial motio etimate. Thi combiatio of iformatio from all level i a mea to tackle the aperture problem ice urface origiatig from coarer level large upport regio will be teeper ad therefore cotribute more to the cumulative urface tha urface from fier level mall upport regio. I additio a the cale depedecy of the SS wa elimiated through the itroductio of the ε factor i equatio 3.38 thi mea that i fact the fier level have greater weight allowig for icreaig adaptivity to local variatio i the motio field. ε factor i equatio 3.38 itead Notice that i ome give referece there i o + CS f term i equatio 3.7 i multiplied by a weightig factor of 4 4λ. Thi term i choe to top level depedecy of P factor give by equatio The effect i therefore to give the fier level iformatio a greater weight i the fial motio etimate Coare-to-Fie Approache Whe combiig iformatio from two ubequet level oe of two approache may be ued. The firt approach i the o-refiig trategy where the fier level etimate S i + computed idepedetly of the previou level etimate CS. The ecod approach i the refiig trategy where the previou level etimate i ued to warp the CWT coefficiet of the referece image at the ext fier level uig equatio 3.4 to produce ew coefficiet 3.7 f f Therefore the field S i computed uig the warped coefficiet f i place of the origial. I additio due to thi warpig a tralatio of the origi back to + i eceary before additio to the previou etimate CS. For a better udertadig the refiig trategy ca be tated a follow Etimate coefficiet { f =...6} uig equatio 3.4 to 3.4. Form S f + f by uig the iterpolated coefficiet itead of the iteger idexed oe 43

57 Fid parameter of valid S f from thoe of S f + f i.e. tralate origi back to Add valid Traform S to caled m f CS to { α β... } + CS to form form. CS equatio 3.7 Itead of the taircae kerel ued i equatio 3.46 the iterpolatio i 3.7 i performed uig a four-tap widowed-ic kerel h f π co k f iπ k f k = 3.73 k f π k f k = -... which Magarey[9] howed to provide better accuracy. The mai advatage of the refiig trategy i that uiform motio of large-cale feature ca be foud with high accuracy. However becaue etimatio ca oly refie valid previou level etimate ay motio of maller cale feature idepedet of large-cale motio will ot be detected. The o-refiig trategy allow cale-depedet motio to be etimated while retaiig the ability of coare cale etimate to propagate dow the pyramid i the abece of clear fier cale iformatio. Coequetly i cae where the motio of mall feature may be aumed to broadly follow that of the larger uch a tereo matchig the refiig trategy i likely to be uitable. However i video codig oe may ot make uch a aumptio ad the o-refiig trategy may be preferable. Aother advatage of the refiig trategy i about the maximum diparity value to be etimated with accuracy. ue to the limit o the rage of f value impoed by the ue of the taircae model whe computig the SS the maximum rage of etimated f value i retricted to.5.5 max mi max max i each directio for the o-refiig trategy ad for the refiig trategy. Coequetly refiig trategy icreae maximum diparity value to be etimated with accuracy. But ay motio of maller cale feature idepedet of large-cale motio ca ot be detected. 44

58 3...4 Curvature Correctio It ha bee tated previouly that flat urface idicate low cofidece i the motio etimate while teep urface idicate high cofidece. I additio coarer level etimate are more aperture-free tha fier level due to their large regio of upport. I a hierarchical approach coarer level etimate hopefully aperture free mut be paed dow to apertureaffected etimate at lower level where aperture-free etimate aimed to determie the parallel compoet of the motio. The curvature of the S urface parallel to the edge determie how much the compoet of motio i that directio beig paed dow from the coarer level i chaged. To allow the parallel compoet of the coarer level etimate to propagate uchaged dow the pyramid aperture-affected ellipe at a fie level hould be ifiitely eccetric i.e. ratio of maor to mior axe i ifiite ad parallel to the uderlyig edge idicatig zero curvature i that directio. A ca be ee from Figure 3. each S ha ifiite eccetricity. Coequetly if a edge i preet oe of the ix ubbad each havig differet orietatio will be excited ad the um S will have ifiite eccetricity. But i practice the CWT wavelet filter have o-zero badwidth ad ay edge will iduce activity i more tha oe ubbad producig elliptical cotour ad therefore fiite eccetricity. The variatio of eccetricity with edge agle i depedet o the propertie of the origial bai pair { h h } - badwidth ad ceter frequecie. If a Gabor filter pair i choe which repod to edge of all orietatio with ellipe of ear-cotat eccetricity it will be poible to correct the curvature of edge-domiated urface to give them very large eccetricity while leavig all other urface uchaged. I thi way the parallel compoet of motio at fie-level aperture-affected ubpel ca be weakeed allowig the more certai parallel compoet of coare-level etimate to propagate uchaged. For thi purpoe a circular bowl with the ame miimum locatio f of ubtracted from all S. The modified quadratic urface i give by S i S f = S f f f f f 3.74 where [ 9] i the radiu of the circular bowl. The locatio ad height δ of the miimum of S are uaffected by curvature correctio. The radiu i choe o that all urface with eccetricity e et for ome e t I [9] Magarey howed that 3.5 i a 45

59 uitable value for e t have very large eccetricity after correctio. The derivatio of i give i [9] a α + β mi. 98λ e t + = 3.75 where λ i defied a α β λ = α + β γ Coequetly curvature correctio i applied to the etire field of urface combiig with previou level urface CS +. S before Cofidece Meaure The algorithm will produce etimate of motio except ome pecial cae ee ectio regardle of tetig the cofidece of etimatio. It i obviou that paig ucofidet etimate due to the break dow of the tralatio model to lower level i a hierarchical maer will caue fale matche. So a cofidece meaure mut be defied ad the etimate which doe ot atify the cofidece criteria mut ot be paed to fier level. The cofidece meaure adopted by Magarey et al. i baed o the weighted um of perpedicular ditace from the ix miimum lie give by each of the S at the miimum poit f i.e. the reidual r 6 = P = T θ f θ 3.77 where θ i the phae gradiet. I the ideal tralatioal model cae all ix cotrait lie coicide ad the reidual i zero. O the other had if the ix ubbad coit of urelated igal the ix lie are dipered ad the reidual i o-zero. Thi reidual may be computed from a quatity called the dicrepacy 46

60 = 6 = P 3.78 ad from the urface miimum value δ give by equatio 3.63 a r = δ 3.79 E where E = 6 = P 3.8 Fially the cofidece meaure i defied a C r = 3.8 The reao for thi defiitio i o that the highet value of C idicate the mot certai etimate. The cofidece meaure alway lie i the rage [ ] due to the coditio that reidual i alway poitive Filter Coefficiet ad Correpodig Gabor Parameter Oe of the rotatio ivariat RI filter pair { h h } utilied i thi work correpod to two 4-tap complex filter with coefficiet h [ i 4 i 4 + i ] h = i [ i 5 + i 5 + i ] 4 = 3.83 i ad correpodig Gabor parameter are = =. 5 w = π 6 w =.76 π σ =.97 σ =. 7 a 47 =. ad a =. 43i. The ecod Gabor filter pair correpod to two 8-tap complex filter with coefficiet 47

61 [. i 5i 4 4 i 35 9i i i + 5i. ] h + = i 3.84 h 3.85 [. i + 5i 4 4 i i i 4 4 i + 5i. ] = i ad correpodig Gabor parameter are = 4 =. 5 w = π 6 w =.83 π σ =.7 σ =. 7 a 39 =. ad a =. 39. The prefilter f dicued before i a 3-tap complex filter with coefficiet [ 5 i] 5 f = i Summary Thi chapter ha decribed the CWT-baed motio etimatio algorithm of Magarey ad Kigbury[ 3 7 9]. The algorithm i baed o a phae-matchig criteria implemeted i a hierarchical maer i.e. it work by matchig the traform phae at each level. Thi i achieved through the defiitio of a quatity called the ubbad quare differecess at each ubpel for all orietatio ubbad. The ix orieted SS urface are added to produce a igle matchig urface whoe miimum locatio i the motio etimate at that ubpel. For the accuracy of etimate at each ubpel ome improvemet method to the origial method i itroduced uch a cofidece threholdig curvature correctio ad refiig trategy. I the ext chapter the algorithm will be aalyzed ad experimeted. 48

62 CHAPTER 4 SIMULATIONS I previou chapter the ME algorithm baed o complex dicrete wavelet traform i aalyzed ad implemeted. The algorithm ue the relatio betwee patial tralatio of image iteity with phae hift i Fourier domai i order to etimate motio i a hierarchical maer. The ME algorithm ha everal choice of parameter ad optio for obtaiig more accurate reult for variou tereo image pair. I thi chapter the effect of parameter choice ad optio icluio o accuracy of ME algorithm will be aalyzed ad experimeted. The wide rage of optio ad parameter that will be aalyzed i give below: CWT Aalyi Stage Optio ad Parameter Uage of 8-tap Gabor filter or 4-tap Gabor filter. Optio ad Parameter I A Level Cofidece Threhold Curvature Correctio Parameter e t Optio ad Parameter Betwee Level Number of Pyramid Level max Choice of Fiet Level mi Coare-to-fie Iterpolatio Type 49

63 4. Tet ata I all motio etimatio tet ythetic image equece are ued i order to quatify the accuracy of the propoed algorithm with repect to true motio field. Sice ythetic image equece are obtaied by a camera motio relative to a tatic cee true motio vector of the image equece i readily kow. There i a cotrait o iput image comig from CWT decompoitio tree. The dimeio of the iput image mut be a multiple of origial image may be exteded to the earet multiple of max. Becaue of thi cotrait the max or a portio of the origial max image with dimeio a multiple of may be ued. But i the ecod techique there might be igificat iformatio lo. So firt techique i commoly ued. A brief decriptio of ythetic image equece that will be ued i tetig the algorithm i give below. Note that the choice of the tree equece i due to the property that motio ca be well defied ice the picture i ot highly or weakly featured ad the choice of Yoemite equece i to tet algorithm agait dicotiuou motio field. Sythetic image ca be obtaied from ftp addre ftp.cd.uwo.ca/pub/viio Tralatig TreeT.Tree: Thi ad the followig equece are obtaied by imulatig camera motio relative to a cee featurig a tree o a mooth backgroud. The curret frame how i Figure 4.a i the 6x6 exteded verio of the th tralatig tree image which i a horizotally diplaced verio of the 8 th tralatig tree equece. The true motio field i compoed of vector with magitude ragig from 3.46 d The aim to ue thi equece i to tet ME algorithm for horizotal tralatioal motio. Rotatig TreeR.Tree: Thi equece i obtaied by rotatig the th tralatig tree image ad rotatig degree i couter-clock wie directio. The ize of the image are exteded to 6x6. The aim of thi equece i to tet ME algorithm agait d-tralatioal motio i.e. horizotal ad vertical tralatioal motio. ivergig Tree.Tree: The equece imulate camera motio toward the cee i.e. a aalog zoomig operatio. The curret ad referece frame are 44x44 exteded verio of the th ad 9 th divergig tree image. The true motio field i compoed of vector ragig from i the ceter to pixel o edge. The aim to ue thi equece i to tet ME algorithm for diverget motio. YoemiteY.Seq.: The equece i a imulated fly-through of Yoemite Natioal Park produced by Ly Quam at Staford Reearch Ititute which i how i Figure 4.b. The curret ad referece frame are 56x3 exteded verio of the th ad 9 th Yoemite 5 hor

64 Figure 4. a Curret frame of ythetic tree equece b Curret frame of Yoemite equece equece. The motio i of two kid: diverget a the camera move alog the lie of the valley ad tralatig a the cloud tralate from left to right at pel/frame. The maximum diplacemet i pel at the lower left corer. The aim to ue thi equece i to tet ME algorithm for dicotiuou motio field. Note that by thee four equece ME algorithm will be teted for motio i all poible dimeio ad dicotiuou motio field. I Figure 4. the correpodig true motio vector field for tree ad Yoemite equece are give where kowledge of true motio vector i eceary to quatify the accuracy of the etimate produced by the algorithm. 4. Error Meauremet Sice ythetic equece are ued for tetig the algorithm true motio vector of the image are kow. Thi eable error meauremet of etimated optical flow field with repect to kow true motio field. If image equece are thought of a et of lide i a proector a 3d diplacemet etimate = with third dimeio i et to coect each pel to it correpodig pel i the ext lide. Uig thi 3d diplacemet etimate a error meaure ca be defied. Oe of the commo error meauremet choice i the relative error defied by equatio 4.. 5