LOW POWER H.264 VIDEO COMPRESSION HARDWARE DESIGNS. by Mustafa Parlak

Transkript

1 LOW POWER H.264 VIDEO COMPRESSION HARDWARE DESIGNS by Mustafa Parlak Submitted to te Graduate Scool of Engineering and Natural Sciences in artial fulfillment of te requirements for te degree of Doctorate of Pilosoy Sabancı Uniersity February 29

2 LOW POWER H.264 VIDEO COMPRESSION HARDWARE DESIGNS APPROVED BY: Assist. Prof. Dr. İlker Hamzaoğlu (Tesis Suerisor). Assist. Prof. Dr. Ayan Bozkurt. Assist. Prof. Dr. Müjdat Çetin. Prof. Dr. Günan Dündar. Assist. Prof. Dr. Hakan Erdoğan. DATE OF APPROVAL:. II

3 To my Moter, Fater, Broters and Sisters To my beloed wife Neslian and our future cildren III

4 ACKNOWLEDGEMENT I would like to tank my suerisor, Dr. İlker Hamzaoğlu for all is guidance, suort, and atience trougout my PD study. It as been a great onor for me to work under is guidance. I would also like to tank my tesis committee members Dr. Ayan Bozkurt and Dr. Müjdat Çetin for teir aluable comments on te dissertation, and Dr. Günan Dündar and Dr. Hakan Erdoğan for articiating in my tesis jury. My secial tanks to System-on-Ci Design & Test grou members, articularly Yusuf Adıbelli and Özgür Taşdizen for teir collaboration and el during my PD study. My sincere tanks to all my friends and colleagues in Sabancı Uniersity including Memet Özdemir, Aliser Kolmato, Ünal Şen and İbraim İnanç. I areciate teir friendsi and el wic made my life easier and more leasant during my PD study. I am articularly grateful to my arents and my wife, Neslian, for teir constant suort, encouragement, assistance and atience. Witout tem, tis study would neer ae been ossible. I would like to tank Sabanci Uniersity for suorting tis researc. I would also like to tank Scientific and Tecnological Researc Council of Turkey (TUBITAK) for suorting tis researc under te contract 6E53. IV

5 LOW POWER H.264 VIDEO COMPRESSION HARDWARE DESIGNS Mustafa Parlak 2 ABSTRACT Video comression systems are used in many commercial roducts suc as digital camcorders, cellular ones and ideo teleconferencing systems. H.264 / MPEG4 Part, te recently deeloed international standard for ideo comression, offers significantly better ideo comression efficiency tan reious international standards. Howeer, tis coding gain comes wit an increase in encoding comlexity and terefore in ower consumtion. Since ortable deices oerate wit battery, it is imortant to reduce ower consumtion so tat te battery life can be increased. In addition, consuming excessie ower degrades te erformance of integrated circuits, increases ackaging and cooling costs, reduces te reliability and may cause deice failures. Terefore, ower consumtion is an imortant design metric for ideo comression ardware. In tis tesis, we roose low ower ardware designs for Deblocking Filter (DBF), intra rediction and intra mode decision arts of an H.264 ideo encoder. Te roosed ardware arcitectures are imlemented in Verilog HDL and maed to Xilinx Virtex II FPGA. We erformed detailed ower consumtion analysis of FPGA imlementations of tese ardware designs using Xilinx XPower tool. We also measured te ower consumtions of DBF ardware imlementations on a Xilinx Virtex II FPGA and tere is a good matc between estimated and measured ower consumtion results. We ten worked on decreasing te ower consumtion of FPGA imlementations of tese H.264 ideo comression ardware designs by reducing switcing actiity using Register Transfer Leel (RTL) low ower tecniques. We alied seeral RTL low ower tecniques suc as clock gating and glitc reduction to tese designs and quantified teir imact on te ower consumtion of te FPGA imlementations of tese designs. We roosed noel comutational comlexity and ower reduction tecniques wic aoid V

6 unnecessary calculations in DBF, intra rediction and intra mode decision arts of an H.264 ideo encoder. We quantified te comutation reductions acieed by te roosed tecniques using H.264 Joint Model software encoder. We alied tese tecniques to roosed ardware designs and quantified teir imact on te ower consumtion of te FPGA imlementations of tese designs. VI

7 DÜŞÜK GÜÇ KULLANIMLI H.264 VİDEO SIKIŞTIRMA DONANIM TASARIMLARI Mustafa Parlak 3 ÖZET Video sıkıştırma sistemleri, dijital kameralar, ce telefonları e ideo telekonferans sistemleri gibi bir çok ticari üründe kullanılmaktadır. Yakın tarite geliştirilmiş uluslararası bir standart olan H.264 / MPEG4 Part, kendinden önceki standartlara göre belirgin şekilde daa iyi sıkıştırma erimi sağlamaktadır. Ancak,bu kodlama kazancı esalama karmaşıklığı e güç tüketimi artışını beraberinde getirmektedir. Taşınabilir ciazlar il ile çalıştığı için, güç tüketimini azaltmak il ömrünün uzamasını sağlayacaktır. Bunun yanında aşırı güç tüketimi, entegre derelerin erformansını düşürür, aketleme e soğutma maliyetlerini arttırır, dayanıklılığını azaltır e bozulmalarına sebe olabilir. Bu nedenle,güç tüketimi, ideo sıkıştırma donanımları için önemli bir tasarım ölçüsüdür. Bu tezde, H.264 Blok Giderici Filtre (BGF), çerçee içi öngörü e çerçee içi ki seçimi algoritmaları için düşük güç kullanımlı donanım tasarımları önerildi. Önerilen donanım mimarileri Verilog HDL ile gerçeklendi e Xilinx Virtex II FPGA ye sentezlendi. Xilinx XPower yazılımı kullanılarak bu donanımların FPGA gerçeklemelerinin detaylı güç tüketim analizleri yaıldı. Ayrıca Xilinx Virtex II FPGA üzerinde çalışan BGF donanımının güç tüketimi ölçüldü e tamin edilen güç tüketimi ile ölçülen güç tüketimi arasında yakın sonuçlar elde edildi. Daa sonra H.264 ideo sıkıştırma donanım tasarımlarının FPGA gerçeklemelerinin güç tüketimleri saklayıcı aktarma (RTL) seiyesinde düşük güç teknikleri ile anatarlama aktiiteleri düşürülerek azaltılmaya çalışıldı. Bu donanımlara saat kaılama, küçük sıçramaları azaltma gibi RTL seiyesinde düşük güç teknikleri uygulandı e bu tekniklerin bu donanımların FPGA içindeki güç tüketimleri üzerindeki etkileri belirlendi. Ayrıca bu tezde H.264 ideo kodlayıcıda bulunan BGF, çerçee içi öngörü e çerçee içi ki seçimi modüllerindeki gereksiz esalamaları engelleyen, özgün sayısal karmaşıklık e güç VII

8 tüketimi azaltıcı teknikler önerildi. Önerilen tekniklerin esalama miktarında yatığı azalma H.264 referans yazılımı (JM) kullanılarak belirlendi. Bu teknikler önerilen donanım tasarımlarına uygulandı e bu tekniklerin bu donanımların FPGA içindeki güç tüketimleri üzerindeki etkileri belirlendi. VIII

9 4 TABLE OF CONTENTS ACKNOWLEDGEMENT... IV 2 ABSTRACT... V 3 ÖZET... VII 4 TABLE OF CONTENTS... IX 6 LIST OF FIGURES... XI 7 LIST OF TABLES... XIII CHAPTER I... INTRODUCTION.. H.264 Video Comression Standard....2 Low Power Hardware Design Tesis Contributions Tesis Organization... 2 CHAPTER II... LOW POWER H.264 DEBLOCKING FILTER HARDWARE DESIGNS Oeriew of H.264 Adatie Deblocking Filter Algoritm Proosed Hardware Arcitectures Imlementation Results ARM Versatile / PB926EJ-S Deeloment Board Imlementation Power Consumtion Results IX

10 2.6 A Noel Comutational Comlexity Reduction Tecnique for H.264 Deblocking Filter CHAPTER III LOW POWER H.264 INTRA PREDICTION HARDWARE DESIGNS H.264 Intra Prediction Algoritm Oeriew Proosed Comutational Comlexity and Power Reduction Tecnique Proosed Intra Prediction Hardware Arcitectures Power Consumtion Analysis CHAPTER IV LOW POWER H.264 INTRA MODE DECISION HARDWARE DESIGNS Hadamard Transform Proosed Comutational Comlexity Reduction Tecnique HT of Predicted Blocks by Intra 4x4 Modes HT of Predicted Blocks by Intra 6x6 and 8x8 Horizontal, Vertical and DC Modes HT of Predicted Blocks by Intra 6x6 and 8x8 Plane Mode Comutation Reduction for Residue Calculations Comutation Reduction Results Proosed 6x6 Intra Mode Decision Hardware Arcitectures Power Consumtion Analysis... 5 CHAPTER V... 5 CONCLUSIONS AND FUTURE WORK 5 6 BIBLIOGRAPHY X

11 6 7 LIST OF FIGURES Figure. H.264 Encoder Block Diagram... 3 Figure.2 H.264 Decoder Block Diagram... 4 Figure.3 ITRS 25 Projection of Maximum Allowable Power Consumtion for ICs. 6 Figure 2. Illustration of H.264 DBF Algoritm... 5 Figure 2.2 Edge Filtering Order in a MB Secified in H.264 Standard... 5 Figure 2.3 H.264 Deblocking Filter Algoritm... 6 Figure 2.4 Proosed DBF Hardware Block Diagram... 8 Figure 2.5 Proosed DBF Hardware Dataat... 9 Figure 2.6 Processing Order of 4 4 Blocks by IT/IQ Module... 2 Figure 2.7 Proosed Noel Edge Filtering Order... 2 Figure 2.8 4x4 Blocks Stored in LUMA and CHROMA SRAMs Figure 2.9 ARM Versatile / PB926EJ-S Deeloment Enironment and Power Measurement Setu Figure 2. Integration of Deblocking Filter Hardware into ARM Versatile Board Figure 2.Unfiltered Video Frame Figure 2.2 Te Same Frame Filtered by H.264 Deblocking Filter Algoritm Figure 3. A 4x4 Luma Block and Neigboring Pixels Figure 3.2 4x4 Luma Prediction Modes Figure 3.3 Examles of Real Images for 4x4 Luma Prediction Modes Figure 3.4 Prediction Equations for 4x4 Luma Prediction Modes XI

12 Figure 3.5 6x6 Luma Prediction Modes Figure 3.6 Examles of Real Images for 6x6 Luma Prediction Modes Figure 3.7 Prediction Equations for 6x6 Luma Prediction Modes... 5 Figure 3.8 Croma Comonent of a MB and its Neigboring Pixels Figure 3.9 Prediction Equations for 8x8 Croma Prediction Modes Figure 3. Four Pixel Grous of Neigboring Pixels of a MB Figure 3. 4x4 Intra Prediction Hardware Arcitecture Figure 3.2 6x6 Intra Prediction Hardware Arcitecture Figure 3.3 8x8 Intra Prediction Hardware Arcitecture Figure 4. Formation of DC Block for Intra 6x6 Prediction Modes Figure 4.2 SATD Calculation for Eac 4x4 Block Figure 4.3 Addition Oerations Performed by Intra Prediction and Mode Decision Figure 4.4 Fast HT Algoritm for a 4x4 Block Figure 4.5 Hadamard Transform of Vertical, Horizontal and DC Modes... 8 Figure 4.6 6x6 MB and its Neigboring Pixels... 9 Figure 4.7 Rate Distortion Cures of te Original SATD Mode Decision and SATD Mode Decision wit Proosed Tecnique for Moter&Daugter (M&D), Crew and Foreman Figure 4.8 Rate Distortion Cures of te Original SATD Mode Decision and SATD Mode Decision wit Proosed Tecnique for Soccer, Football and Mobile Figure 4.9 Proosed Hardware for Original Intra 6x6 Mode Decision... 2 Figure 4. Proosed Hardware for Intra 6x6 Mode Decision wit Proosed Tecnique... 3 XII

13 8 LIST OF TABLES Table 2. Conditions tat Determine BS... 7 Table 2.2 FPGA Resource Usage and Clock Frequency After Place and Route Table 2.3 DBF Hardware Comarison Table 2.4 Power Consumtion of DBF Hardware Imlementations at 5 MHz Table 2.5 Power Consumtion Comarison of Block SelectRAM and Distributed SelectRAM Table 2.6 Imact of Clock Gating on Dataat Power Consumtion Table 2.7 Imact of Glitc Reduction on Dataat Power Consumtion Table 2.8 Power Consumtion Estimations and Measurements of DBF_6 6 and DBF_4 4 Hardware at 34 MHz... 3 Table 2.9 DBF Modes Table 2. Equations for Mode 6 and teir Simlified Versions wen 2===q=q=q Table 2. Equations for Mode 6 and teir Simlified Versions wen ==q=q.. 32 Table 2.2 Te Amount of Comutation Required by DBF Mode For Different Equal Pixel Combinations Table 2.3 Te Amount of Comutation Required by DBF Mode For Different Equal Pixel Combinations Table 2.4 Te Amount of Comutation Required by DBF Mode 2 For Different Equal Pixel Combinations Table 2.5 Te Amount of Comutation Required by DBF Mode 3 For Different Equal Pixel Combinations XIII

14 Table 2.6 Te Amount of Comutation Required by DBF Mode 5 For Different Equal Pixel Combinations Table 2.7 Te Amount of Comutation Required by DBF Mode 6 For Different Equal Pixel Combinations Table 2.8 Te Amount of Comutation Required by DBF Mode 7 For Different Equal Pixel Combinations Table 2.9 Te Amount of Comutation Required by DBF Mode 4 For Different Equal Pixel Combinations Table 2.2 Te Amount of Comutation Required by DBF Mode Table 2.2 Number of Occurrences of Different Equal Pixel Combinations for DBF Mode Table 2.22 Comutation Reduction Results... 4 Table 2.23 Comarison Oeread... 4 Table 3. Aailability of 4x4 Luma Prediction Modes Table 3.2 Aailability of 6x6 Luma Prediction Modes... 5 Table 3.3 Aailability of 8x8 Luma Prediction Modes Table 3.4 4x4 Intra Modes and Corresonding Neigboring Pixels Table 3.5 Percentage of 4x4 Intra Prediction Modes wit Equal Neigboring Pixels Table 3.6 Comutation Amount of 4x4 Intra Modes Table 3.7 Intra 4x4 Modes Comutation Reduction Results Table 3.8 Percentage of 6x6 Intra Prediction Modes wit Equal Neigboring Pixels 6 Table 3.9 Percentage of 8x8 Intra Prediction Modes (Croma CB, CR) wit Equal Neigboring Pixels... 6 Table 3. Comutation Amount of Intra 6x6 and Intra 8x8 Modes... 6 Table 3. Intra 6x6 Comutation Reduction Results Table 3.2 Intra 8x8 (Croma CB, CR) Comutation Reduction Results XIV

15 Table 3.3 FPGA Resource Usages of Original Intra Prediction Hardware and Intra Prediction Hardware wit Proosed Tecnique Table 3.4 Power Consumtion Reduction of Intra 4x4 Prediction Hardware (QP=28) 69 Table 3.5 Power Consumtion Reduction of Intra 4x4 Prediction Hardware (Q=35).. 7 Table 3.6 Power Consumtion Reduction of Intra 4x4 Prediction Hardware (Q=42).. 7 Table 3.7 Power Consumtion Reduction of Intra 6x6 Prediction Hardware (QP=28)... 7 Table 3.8 Power Consumtion Reduction of Intra 6x6 Prediction Hardware (QP= 35)... 7 Table 3.9 Power Consumtion Reduction of Intra 6x6 Prediction Hardware (QP= 42) Table 3.2 Power Consumtion Reduction of Intra 8x8 Prediction Hardware (QP=28) 72 Table 3.2 Power Consumtion Reduction of Intra 8x8 Prediction Hardware (QP=35) 73 Table 3.22 Power Consumtion Reduction of Intra 8x8 Prediction Hardware (QP=42) 73 Table 4. Pre-calculated Values for DDL Prediction Mode Table 4.2 DDL Mode Prediction Calculations Using Pre-calculated Values Table 4.3 Pre-calculated Values for DDR Prediction Mode Table 4.4 DDR Mode Prediction Calculations Using Pre-calculated Values Table 4.5 Pre-calculated Values for VR Prediction Mode Table 4.6 VR Mode Prediction Calculations Using Pre-calculated Values Table 4.7 Pre-calculated Values for HUP Prediction Mode Table 4.8 HUP Mode Prediction Calculations Using Pre-calculated Values Table 4.9 Comutation Reductions for Intra Prediction Modes Table 4. Aerage PSNR Comarison of te Original SATD Mode Decision wit Proosed Tecnique Table 4. Power Consumtion Reduction of Intra 6x6 Mode Decision Hardware (QP=42)... 4 XV

16 CHAPTER I INTRODUCTION. H.264 Video Comression Standard Video comression systems are used in many commercial roducts, from consumer electronic deices suc as digital camcorders, cellular ones to ideo teleconferencing systems. Tese alications make te ideo comression ardware deices an ineitable art of many commercial roducts. To imroe te erformance of te existing alications and to enable te alicability of ideo comression to new real-time alications, recently, a new international standard for ideo comression is deeloed. Tis new standard, offering significantly better ideo comression efficiency tan reious ideo comression standards, is deeloed wit te collaboration of ITU and ISO standardization organizations. Hence it is called wit two different names, H.264 and MPEG4 Part []. H.264 ideo coding standard as a muc iger coding efficiency (caable of saing u to %5 bit rate at te same leel of ideo quality) tan te reious standards [2]. Due to its ig coding efficiency and due to its flexibility and robustness to different

17 communication enironments, in te near future, H.264 is exected to be widely used in many alications suc as digital TV, DVD, ideo transmission in wireless networks, and ideo conferencing oer te internet. Te uman isual system aears to distinguis scene content in terms of brigtness and color information indiidually, and wit greater sensitiity to te details of brigtness tan color [3]. Same as te reious ideo comression standards, H.264 is designed to take adantage of tis by using YCbCr color sace. In YCbCr color sace, eac ixel is reresented wit tree 8-bit comonents called Y, Cb, and Cr. Y, te luminance (luma) comonent, reresents brigtness. Cb and Cr, crominance (croma) comonents, reresent te extent to wic te color differs from gray toward blue and red, resectiely. Since te uman isual system is more sensitie to luma comonent tan croma comonents, H.264 standard uses 4:2: samling. In 4:2: samling, for eery four luma samles, tere are two croma samles, one Cb and one Cr. Te to-leel block diagram of an H.264 ideo encoder is sown in Figure.. As sown in te figure, te ideo comression efficiency acieed in H.264 standard is not a result of any single feature but rater a combination of a number of encoding tools suc as motion estimation, intra rediction and deblocking filter (DBF). Same as te reious ideo comression standards, H.264 standard does not secify all te algoritms tat will be used in an encoder suc as mode decision. Instead, it defines te syntax of te encoded bit stream and functionality of te decoder tat can decode tis bit stream. As sown in Figure., an H.264 encoder as a forward at and a reconstruction at. Te forward at is used to encode a ideo frame and create te bit stream by using intra and inter redictions. Te reconstruction at is used to decode te encoded frame and reconstruct te decoded frame. Since a decoder neer gets original images, but rater works on te decoded frames, reconstruction at in te encoder ensures tat bot encoder and decoder use identical reference frames for intra and inter rediction. Tis aoids ossible encoder decoder mismatces [,3,4]. 2

18 Figure. H.264 Encoder Block Diagram Forward at starts wit artitioning te inut frame into macroblocks (MB). Eac MB is encoded in intra or inter mode deending on te mode decision. In bot intra and inter modes, te current MB is redicted from te reconstructed frame. Intra mode generates te redicted MB based on satial redundancy, wereas inter mode, generates te redicted MB based on temoral redundancy. Mode decision comares te required amount of bits to encode a MB and te quality of te decoded MB for bot of tese modes and cooses te mode wit better quality and bit-rate erformance. In eiter case, intra or inter mode, te redicted MB is subtracted from te current MB to generate te residual MB. Residual MB is transformed using 4x4 and 2x2 integer transforms. Transformed residual data is quantized and quantized transform coefficients are re-ordered in a zig-zag scan order. Te reordered quantized transform coefficients are entroy coded. Te entroycoded coefficients togeter wit eader information, suc as MB rediction mode and quantization ste size, form te comressed bit stream. Te comressed bit stream is assed to network abstraction layer (NAL) for storage or transmission [,3,4]. Reconstruction at begins wit inerse quantization and inerse transform oerations. Te quantized transform coefficients are inerse quantized and inerse transformed to generate te reconstructed residual data. Since quantization is a lossy rocess, inerse quantized and inerse transformed coefficients are not identical to te original residual data. Te reconstructed residual data are added to te redicted ixels in order to create te reconstructed frame. DBF is, ten, alied to reduce te effects of blocking artifacts in te reconstructed frame [,3,4]. 3

19 H.264 intra rediction and mode decision algoritms ae a ery ig comutational comlexity. Because, in order to imroe te comression efficiency, H.264 standard uses many intra rediction modes for a MB and selects te best mode for tat MB using a mode decision algoritm. Te DBF algoritm used in H.264 standard is more comlex tan te DBF algoritms used in reious ideo comression standards. First of all, H.264 DBF algoritm is igly adatie and alied to eac edge of all te 4 4 luma and croma blocks in a MB. Second, it can udate 3 ixels in eac direction tat te filtering takes lace. Tird, in order to decide weter te DBF will be alied to an edge, te related ixels in te current and neigboring 4 4 blocks must be read from memory and rocessed. Because of tese comlexities, te DBF algoritm can easily account for one-tird of te comutational comlexity of an H.264 ideo decoder [4,5]. Reference Frame (F n- ) Reconstructed Frame (F n ) Deblocking Filter Motion Comensation Intra Prediction Inter Prediction Intra Prediction Prediction Inerse Transform Figure.2 H.264 Decoder Block Diagram Inerse Quant NAL Entroy Decoder H.264 decoder is similar to te reconstruction at of H.264 encoder. It receies a comressed bit stream from te NAL as sown in Figure.2. Te bit stream is decoded, inerse quantized and inerse transformed to get residual data. Using te eader information decoded from te bit stream, te decoder creates a rediction block, identical to rediction block generated in reconstruction at of H.264 encoder. Te rediction block is added to te residual block to create te reconstructed block. Blocking artifacts are, ten, remoed from reconstructed block by alying DBF. H.264 as tree rofiles; Baseline, Main, and Extended. Eac rofile as 4 leels. A rofile is a set of algoritmic features and a leel sows encoding caability suc as icture 4

20 size and frame rate. In tis tesis, we use Baseline rofile leel 2.. Baseline rofile as lower latency tan main and extended rofiles, and it is used for wireless ideo alications and ideo conferencing. In Baseline rofile leel 2., ideo is digitized at CIF (352x288) size YCbCr using 4:2: samling at 3 frames er second, and I slices, P slices and context-adatie ariable lengt entroy coding are suorted [,3]..2 Low Power Hardware Design Multimedia alications running on ortable deices ae increased recently and tis trend is exected to continue in te future. Since ortable deices oerate wit battery, it is imortant to reduce ower consumtion so tat battery life can be increased. Terefore, ower consumtion as become a critical design metric for ortable alications. In addition, consuming excessie ower for a long time causes te cis to eat u and degrades te erformance, because transistors run faster wen tey are cool rater tan ot. Excessie ower consumtion also increases ackaging and cooling costs. Excessie ower consumtion also reduces te reliability and may cause deice failures [6]. Reeated cycling from ot to cool sortens te life of a ci by inducing mecanical stress tat can literally tear a ci aart. Hot metal interconnects on te ci are also more suscetible to disintegration because of a enomenon called electromigration. Terefore, tere is an uer bound for allowed ower consumtion in integrated circuits (IC). Te maximum allowable ower consumtion for ICs rojected by International Tecnology Roadma for Semiconductors in 25 is gien in Figure.3 [7]. In te figure, maximum allowable ower for tree tyes of alications is resented; ig-erformance alications for wic a eat sink on te ackage is ermitted, cost-erformance alications for wic economical ower management solutions are used and alications wit ortable battery for wic no cooling system is used. In all cases, total ower consumtion continues to increase, desite te use of a lower suly oltage. Te increased ower consumtion is drien by iger oerating frequencies, iger oerall interconnect caacitance, and exonentially growing number of scaled transistors [7]. Terefore, ower consumtion is an imortant design metric for all alications. 5

21 Figure.3 ITRS 25 Projection of Maximum Allowable Power Consumtion for ICs Field Programmable Gate Arrays (FPGA) consume more ower tan standard cellbased Alication Secific Integrated Circuits (ASIC). FPGAs ae look-u tables and rogrammable switces. Look-u table based logic imlementation is inefficient in terms of ower consumtion and rogrammable switces ae ig ower consumtion because of large outut caacitances. Terefore, ower consumtion is an een more imortant design metric for FPGA imlementations. ICs ae static and dynamic ower consumtion. Static ower consumtion is a result of leakage currents in an IC. Dynamic ower consumtion is a result of sort circuit currents and carging and discarging of caacitances in an IC. Dynamic ower consumtion is roortional to te switcing actiity (α), total caacitance (C L ), suly oltage (V DD ), oerating frequency (f) and sort circuit current (I SC ) as sown in te following equation. Te ower consumtion due to carging and discarging of caacitances is te dominant comonent of dynamic ower consumtion and it can be reduced eiter by decreasing switcing actiity, caacitance, suly oltage or frequency. 6

22 P 2 dyn L DD α C V f I V f (.) SC DD In tis tesis, we focused on decreasing te ower consumtion of FPGA imlementations of H.264 ideo comression ardware by reducing switcing actiity using Register Transfer Leel (RTL) low ower tecniques suc as clock gating [8,9,], glitc reduction [,2] and comutational comlexity reduction [3,4,5]. Te ower consumtion of a digital ardware imlementation on a Xilinx FPGA is estimated using Xilinx XPower tool. Since te switcing actiity is inut attern deendent, in order to estimate te dynamic ower consumtion, timing simulation of te laced and routed netlist of tat ardware imlementation is done for seeral inut atterns using Mentor Graics ModelSim SE 6.c and te signal actiities are stored in a Value Cange Dum (VCD) file. Tis VCD file is used for estimating te ower consumtion of tat ardware using Xilinx XPower tool..3 Tesis Contributions H.264 standard is exected to be used in many alications in te near future. Terefore, in te last few years, ardware arcitectures for imlementing H.264 encoders and decoders for ortable deices, digital TV and DVD recorder alications are started to be deeloed in bot academia and industry. Howeer, since H.264 standard as been recently deeloed, tere are a small number of ublications in te literature about designing ardware arcitectures for H.264 standard [6,7,8,9,2,2]. Tere are een fewer ublications in te literature about low ower ardware arcitectures for H.264 standard [22,23,24]. In tis tesis, we roosed low ower ardware designs for DBF, intra rediction and intra mode decision arts of an H.264 ideo encoder and erformed detailed ower consumtion analysis of FPGA imlementations of tese ardware designs. We also measured te ower consumtions of DBF ardware imlementations on a Xilinx Virtex II FPGA and tere is a good matc between estimated and measured ower consumtion results. We alied seeral RTL low ower tecniques suc as clock gating and glitc 7

23 reduction to tese designs and quantified teir imact on te ower consumtion of te FPGA imlementations of tese designs. We roosed noel comutational comlexity and ower reduction tecniques for DBF, intra rediction and intra mode decision arts of an H.264 ideo encoder. We quantified te comutation reductions acieed by te roosed tecniques using H.264 Joint Model (JM) software encoder ersion 4.. We alied tese tecniques to roosed ardware designs and quantified teir imact on te ower consumtion of te FPGA imlementations of tese designs. We roose two efficient and low ower H.264 DBF ardware imlementations tat can be used as art of an H.264 ideo encoder or decoder for ortable alications [25]. Te first imlementation (DBF_4x4) starts filtering te aailable edges as soon as a new 4x4 block is ready by using a noel edge filtering order to oerla te execution of DBF module wit oter modules in te H.264 encoder/decoder. Oerlaing te execution of DBF ardware wit te execution of te oter modules in te H.264 encoder/decoder imroes te erformance of te H.264 encoder/decoder. Te second imlementation (DBF_6x6) starts filtering te aailable edges after a new 6x6 MB is ready. Bot DBF ardware arcitectures are imlemented in Verilog HDL and bot imlementations are syntesized to.8 µm UMC standard cell library. Bot DBF imlementations can work at 2 MHz and tey can rocess 3 VGA (64 48) frames er second. DBF_4 4 and DBF_6 6 ardware imlementations, excluding on-ci memories, are syntesized to 7.4 K and 5.3 K gates resectiely. Tese gate counts are te lowest among te H.264 DBF ardware imlementations resented in te literature. DBF_6x6 as 36% less ower consumtion tan DBF_4x4 on a Xilinx Virtex II FPGA on an Arm Versatile PB926EJ-S deeloment board. Terefore, DBF_4 4 ardware can be used in an H.264 encoder or decoder for wic te erformance is more imortant, wereas DBF_6 6 ardware can be used in an H.264 encoder or decoder for wic te ower consumtion is more imortant. We roose a noel comutational comlexity and ower reduction tecnique for H.264 DBF algoritm. Tis tecnique aoids unnecessary calculations in DBF algoritm by exloiting satial redundancy resent in te ixels tat will be filtered by DBF algoritm and terefore reduces te ower consumtion of H.264 DBF ardware significantly. If some or all of te ixels tat will be filtered are equal, H.264 DBF 8

24 equations simlify significantly. Since te roosed tecnique uses te subtraction oerations erformed before te filtering rocess to determine weter te ixels tat will be filtered are equal or not, te equality of te ixels are determined wit a ery small oeread. By exloiting te equality of te ixels, te roosed tecnique reduces te amount of addition and sift oerations erformed by H.264 DBF u to 39% and 5% resectiely wit a small comarison oeread. We roose a noel tecnique for reducing te amount of comutations erformed by H.264 intra rediction algoritm and terefore reducing te ower consumtion of H.264 intra rediction ardware significantly witout any PSNR and bit rate loss. Te roosed tecnique erforms a small number of comarisons among neigboring ixels of te current block before te intra rediction rocess. If te neigboring ixels of te current block are equal, te rediction equations of H.264 intra rediction modes simlify significantly for tis block. By exloiting te equality of te neigboring ixels, te roosed tecnique reduces te amount of comutations erformed by 4x4 luminance, 6x6 luminance, and 8x8 crominance rediction modes u to 6%, 28%, and 68% resectiely wit a small comarison oeread. We also imlemented an efficient 4x4 intra rediction ardware including te roosed tecnique using Verilog HDL. We quantified te imact of te roosed tecnique on te ower consumtion of tis ardware on a Xilinx Virtex II FPGA using Xilinx XPower, and it reduced te ower consumtion of tis ardware u to 8.6% [26]. We roose a noel comutational comlexity reduction tecnique for H.264 intra mode decision. Te roosed tecnique exloits te fixed rediction block atterns of intra rediction modes and te distribution roerty of Hadamard transform. Te roosed tecnique reduces te comutational comlexity of Sum of Absolute Transformed Difference (SATD) based intra 4x4, intra 6x6 and intra 8x8 mode decisions by 46%, 64% and 62% resectiely witout any PSNR loss. In addition, it aoids te calculation of intra 6x6 and intra 8x8 lane rediction modes by sligtly modifying SATD criterion used in H.264 reference software (JM) wic sligtly imacts te coding efficiency. It doesn t affect te PSNR for some ideos, it increases te PSNR sligtly for some ideos and it decreases te PSNR sligtly for some ideos. Since lane mode is te most comutationally intensie 6x6 and 8x8 rediction mode, aoiding lane mode 9

25 calculations reduces te comutational comlexity of 6x6 and 8x8 intra rediction algoritm by 8%..4 Tesis Organization Te rest of te tesis is organized as follows. Cater II, first, gies an oeriew of H.264 DBF algoritm. It, ten, resents two efficient low ower H.264 DBF ardware imlementations and te imact of seeral RTL low ower tecniques on tese ardware imlementations. A noel comutational comlexity and ower reduction tecnique for H.264 DBF algoritm is also resented in tis cater. Cater III, first, gies an oeriew of H.264 intra rediction algoritm. It, ten, resents a noel comutational comlexity and ower reduction tecnique for H.264 intra rediction. An efficient H.264 intra rediction ardware imlementing tis tecnique and its ower consumtion analysis is also resented in tis cater. Cater IV, first, gies an oeriew of H.264 intra mode decision algoritm. It, ten, resents a noel comutational comlexity and ower reduction tecnique for H.264 intra mode decision. Cater V resents conclusions and future work.

26 2 CHAPTER II LOW POWER H.264 DEBLOCKING FILTER HARDWARE DESIGNS Te ideo comression efficiency acieed in H.264 standard is not a result of any single feature but rater a combination of a number of encoding tools. As it is sown in te to leel block diagrams of an H.264 encoder and decoder in Figure. and.2., one of tese tools is te adatie DBF algoritm [,3,4,27]. DBF is alied to eac MB, a 6 6 ixel array, after inerse quantization and inerse transform. DBF imroes te isual quality of decoded frames by reducing te isually disturbing blocking artifacts and discontinuities in a frame due to coarse quantization of MBs and motion comensated rediction. Since te filtered frame is used as a reference frame for motion-comensated rediction of future frames, DBF also increases coding efficiency resulting in bit rate saings [27]. Te DBF algoritm used in H.264 standard is more comlex tan te DBF algoritms used in reious ideo comression standards. First of all, H.264 DBF algoritm is igly adatie and alied to eac edge of all te 4 4 luma and croma blocks in a MB. Second, it can udate 3 ixels in eac direction tat te filtering takes lace. Tird, in order to decide weter te DBF will be alied to an edge, te related ixels in te current and neigboring 4 4 blocks must be read from memory and rocessed.

27 Because of tese comlexities, te DBF algoritm can easily account for one-tird of te comutational comlexity of an H.264 ideo decoder [27]. In tis tesis, we roose two efficient and low ower H.264 DBF ardware imlementations tat can be used as art of an H.264 ideo encoder or decoder for ortable alications [28,29]. Te first imlementation (DBF_4 4) starts filtering te aailable edges as soon as a new 4x4 block is ready by using a noel edge filtering order. Te second imlementation (DBF_6 6) starts filtering te aailable edges after a new 6x6 MB is ready. Te execution of DBF_4 4 ardware can be oerlaed wit te execution of te oter modules in an H.264 encoder/decoder muc more tan te execution of DBF_6 6 ardware can be oerlaed wit te execution of te oter modules. Oerlaing te execution of DBF ardware wit te execution of te oter modules in te H.264 encoder/decoder imroes te erformance of te H.264 encoder/decoder. Howeer, because of te nature of te DBF algoritm, control unit and address generation of DBF_6 6 ardware is simler. Terefore, DBF_6x6 ardware as less area and consumes less ower tan DBF_4 4 ardware. Bot DBF ardware arcitectures are imlemented in Verilog HDL and bot imlementations are erified to work correctly in a Xilinx Virtex II FPGA on Arm Versatile PB926EJ-S Deeloment Board. Bot ardware imlementations can work at 67 MHz on a Xilinx Virtex II FPGA and tey can rocess 3 CIF (352x288) frames er second. Bot ardware imlementations can work at 2 MHz wen syntesized to.8 µm UMC standard cell library and tey can rocess 3 VGA (64 48) frames er second. DBF_4 4 and DBF_6 6 ardware imlementations, excluding on-ci memories, are syntesized to 7.4 K and 5.3 K gates resectiely. Te ower consumtions of bot DBF ardware imlementations on a Xilinx Virtex II FPGA are estimated using Xilinx XPower tool. DBF_6 6 as 36% less ower consumtion tan DBF_4 4. Te ower consumtion of DBF_6 6 is furter reduced by 28% by using block SelectRAMs instead of distributed SelectRAMs and by 3.% by using clock gating. Furtermore, ower consumtion of DBF dataat is reduced by 3% using clock gating and by 4.7% using glitc reduction tecnique. Te ower consumtions of 2

28 bot imlementations on a Xilinx Virtex II FPGA are also measured and te measurement results are consistent wit te estimation results. Terefore, tese two H.264 DBF ardware imlementations can be used as art of H.264 ideo encoders or decoders for ortable alications wit different owererformance requirements. DBF_4 4 ardware can be used in an H.264 encoder or decoder for wic te erformance is more imortant, wereas DBF_6 6 ardware can be used in an H.264 encoder or decoder for wic te ower consumtion is more imortant. Seeral ardware arcitectures for real-time imlementation of H.264 DBF algoritm are resented in te literature [6,3,3,32,33]. Tese arcitectures aciee ig erformance at te exense of ig ardware cost. Te gate counts of our H.264 DBF ardware imlementations are te lowest among tese H.264 DBF ardware imlementations. We could not comare ower consumtions of our DBF ardware imlementations wit tese DBF ardware imlementations, since te ower consumtions of tese DBF ardware imlementations are not reorted. We also roose a noel comutational comlexity and ower reduction tecnique for H.264 DBF algoritm. Tis tecnique aoids unnecessary calculations in DBF algoritm by exloiting satial redundancy resent in te ixels tat will be filtered by DBF algoritm and terefore reduces te ower consumtion of H.264 DBF ardware significantly. If some or all of te ixels tat will be filtered are equal, H.264 DBF equations simlify significantly. Since te roosed tecnique uses te subtraction oerations erformed before te filtering rocess to determine weter te ixels tat will be filtered are equal or not, te equality of te ixels are determined wit a ery small oeread. By exloiting te equality of te ixels, te roosed tecnique reduces te amount of addition and sift oerations erformed by H.264 DBF u to 39% and 5% resectiely wit a small comarison oeread. 3

29 2. Oeriew of H.264 Adatie Deblocking Filter Algoritm H.264 adatie DBF algoritm remoes isually disturbing block boundaries created by coarse quantization of MBs and motion comensated rediction. MBs in a frame are filtered in raster scan order. Filtering is alied to eac edge of all te 4x4 luma and croma blocks in a MB as sown in Figure 2.. Te ertical 4 4 block edges in a MB are filtered before te orizontal 4 4 block edges in te order sown in Figure 2.2 []. DBF algoritm for one row/column of a ertical/orizontal edge is sown in Figure 2.3 [27]. Tere are seeral conditions tat determine weter a 4 4 block edge will be filtered or not. Tere are additional conditions tat determine te strengt of te filtering for te 4x4 block edges tat will be filtered. As sown in te Figure 2.3, boundary strengt (BS) arameter, α and β tresold alues and te alues of te ixels in te edge determine te outcomes of tese conditions, and te alues of u to 3 ixels on bot sides of an edge can be canged deending on te outcomes of tese conditions. H.264 DBF algoritm can be diided into eigt modes based on te outcomes of tese conditions as sown in Figure 2.3. H.264 DBF algoritm is adatie in tree leels; slice leel, edge leel and samle leel [,27]. Slice leel adatiity is used to adjust te filtering strengt in a slice to te caracteristics of te slice data. Te filtering strengt in a slice is adjusted by encoder using te offset-a and offset-b arameters. Te α and β tresold alues tat determine weter a 4x4 block edge will be filtered or not and ow strong te filtering will be for an edge are a function of quantization arameter (QP) and tese two offset arameters. Edge leel adatiity is used to adjust te filtering strengt for an edge to te caracteristics of tat edge. Te filtering strengt for an edge is adjusted using te Boundary Strengt (BS) arameter. Eery edge is assigned a BS alue deending on te coding modes and conditions of te 4x4 blocks. Te conditions used for determining te BS alue for an edge between two neigboring 4x4 blocks are summarized in Table 2. [,27]. Te strengt of te filtering done for an edge is roortional to its BS alue. No filtering is done for te edges wit a BS alue of zero, wereas strongest filtering is done for te edges wit a BS alue of four. 4

30 Figure 2. Illustration of H.264 DBF Algoritm Figure 2.2 Edge Filtering Order in a MB Secified in H.264 Standard 5

31 Figure 2.3 H.264 Deblocking Filter Algoritm 6

32 Samle leel adatiity is used to adjust te filtering strengt for an edge to te caracteristics of te ixels in tat edge in order to distinguis te true edges from tose created by quantization. Te filtering strengt for an edge is terefore determined by comaring ixel gradients in tat edge wit α and β tresold alues for tat edge. Table 2. Conditions tat Determine BS Coding Modes and Conditions BS One of te blocks is intra and te edge is a macroblock edge 4 One of te blocks is intra 3 One of te blocks as coded residuals 2 Difference of block motion luma samle distance and Motion comensation from different reference frames Else 2.2 Proosed Hardware Arcitectures Te block diagram of roosed DBF ardware is sown in Figure 2.4. Bot DBF ardware, DBF_4x4 and DBF_6x6, include a dataat, a control unit, one register file and two dual-ort internal SRAMs to store artially filtered ixels. As it can be seen from Figure. and.2, in an H.264 encoder and decoder, DBF module gets its inut, reconstructed MB, from Inerse Transform/Quant (IT/IQ) module. IT/IQ module generates te reconstructed MB, one 4 4 block at a time. A inut buffer, IBUF, is used between IT/IQ and DBF modules to store one reconstructed MB (256 luminance ixels 28 crominance ixels) generated by IT/IQ module. Te dataats of bot DBF ardware imlementations are te same. Te DBF dataat is imlemented as a two stage ieline to imroe te clock frequency and trougut. As sown in Figure 2.5, te first ieline stage includes one 2-bit adder and two sifters to erform numerical calculations like multilication and addition. Te second ieline stage includes one 2-bit comarator, seeral two s comlementers and multilexers to determine conditional branc results. 7

33 Figure 2.4 Proosed DBF Hardware Block Diagram DBF_4 4 ardware starts filtering aailable edges as soon as a new 4x4 block is ready by using a noel edge filtering order we roosed. Tere are blocks in a MB and tey are rocessed by IT/IQ module in te order sown in Figure 2.6 []. Te roosed noel edge filtering order for a MB is sown in Figure 2.7. Te idea beind tis noel order is tat after a new 4 4 block is ready start filtering te edges tat can be filtered witout iolating te filtering order secified in te H.264 standard []. After te first 4 4 block in a MB is rocessed and loaded into IBUF by IT/IQ module, DBF module can only filter edge witout iolating te filtering order secified in H.264 standard. After te second 4 4 block is loaded into IBUF, DBF module can filter edge 2 and edge 3, and so on. Te execution of DBF_4 4 ardware can be oerlaed wit te execution of te oter modules in an H.264 encoder/decoder muc more tan te execution of DBF_6 6 ardware can be oerlaed wit te execution of te oter modules. Oerlaing te execution of DBF ardware wit te execution of te oter modules in te H.264 encoder/decoder imroes te erformance of te H.264 encoder/decoder. 8

34 Figure 2.5 Proosed DBF Hardware Dataat 9

35 DBF_6x6 ardware uses te same edge filtering order secified in te H.264 standard. Since tis edge filtering order as a regular attern, control unit and address generation of DBF_6 6 ardware is simler. Terefore, DBF_6x6 ardware as less area and consumes less ower tan DBF_4 4 ardware. Tere are tree on-ci memories in bot DBF ardware imlementations. A register file, SPAD, is used to store artially filtered ixels in a 6 6 MB until all te edges in tis MB are fully filtered. Since SPAD is te most frequently accessed memory in te DBF ardware, we reduced te number of access to SPAD by adding two registers in dataat to store some of te temorary results. In te M N frame sown in Figure 2.8, squares reresent 6x6 MBs and eac MB as sixteen 4 4 blocks. In order to filter a MB, its uer and left neigboring 4 4 blocks, sown as saded small squares in Figure 2.8, sould be aailable. Since our DBF ardware gets its inut MB from IT/IQ ardware and it does not ae access to off-ci frame memory, te uer 4 4 blocks of all MBs in a row of te frame, sown as ligtly saded small squares in Figure 2.8, and te left 4 4 blocks of te current MB, sown as darkly saded small squares in Figure 2.8, ae to be stored in on-ci local memory. Te left 4x4 blocks are stored in SPAD. Te uer 4x4 luminance and crominance blocks are stored in te 48 8 LUMA SRAM and 74 8 CHROMA SRAM memories sown in Figure 2.4 resectiely. For a CIF size ideo, = 48 8 memory is needed for storing uer 4x4 luminance blocks and 4x88x84x88x8 = 74 8 memory is needed for storing uer 4x4 crominance blocks. Figure 2.6 Processing Order of 4 4 Blocks by IT/IQ Module 2

36 Figure 2.7 Proosed Noel Edge Filtering Order Te DBF ardware imlementations in te literature use off-ci memory for storing tese neigboring 4 4 blocks [6,3,3,32,33]. Since accessing on-ci SRAMs consumes less ower tan accessing off-ci memory, using on-ci SRAMs for storing tese neigboring 4 4 blocks reduces ower consumtion of our DBF ardware imlementations. Transose ixel arrays are used to transose te orizontal aligned ixels into ertical aligned ositions in seeral DBF ardware imlementations in te literature [3,3,32,33]. Since te memories used in our DBF ardware imlementations are 8-bit wide, any ixel stored in memory can directly be accessed, terefore, tere is no need for transosing one row of eigt ixel data into one column of eigt ixel data. Not using a transose ixel array reduces area of our DBF ardware imlementations. Te edges, 2, 3, 4, 7, 8, 9, 2, 33, 34, 37, 38, 4, 42, 45 and 46 of a MB sown in Figure 2.3 are not filtered if tis MB is located in te uer or te left frame boundary. Tis is not te case for te MBs located inside te frame. Tis causes an 2

37 irregularity and, terefore, increases te comlexity of te control unit. In order to aoid tis irregularity and terefore simlify te control unit, we ae extended te frames at te uer and left frame boundaries for 4 ixels in det as sown in Figure 2.8. We assigned zero to tese ixels and assigned zero to te BS alues of tese edges in order to aoid filtering tese edges witout causing an irregularity in te control unit. Figure 2.8 4x4 Blocks Stored in LUMA and CHROMA SRAMs 22

38 2.3 Imlementation Results Te roosed DBF ardware arcitectures are imlemented in Verilog HDL. Te imlementations are erified wit RTL simulations using Mentor Graics ModelSim SE. RTL simulation results matced te results of a MATLAB model of te H.264 adatie DBF algoritm. Te Verilog RTL designs are syntesized to a 2V8ff57 Xilinx Virtex II FPGA wit seed grade 5 using Mentor Graics Precision RTL 25b. Te resulting netlists are laced and routed to te same FPGA using Xilinx ISE 8.2i. DBF_4x4 ardware works at 67 MHz and it takes 5248 clock cycles in te worstcase for DBF_4 4 ardware to rocess a MB. Te FPGA imlementation can rocess a CIF (352x288) frame in 3.9 ms (396 MB * 5248 clock cycles er MB * 4.9 ns clock cycle = 3.9 ms). Terefore, it can rocess /3.9 = 32 CIF frames er second. DBF_6x6 ardware works at 72 MHz and it takes 5376 clock cycles in te worst-case for DBF_6 6 ardware to rocess a MB. Te FPGA imlementation can rocess a CIF (352x288) frame in 29.6 ms (396 MB * 5376 clock cycles er MB * 3.9 ns clock cycle = 29.6 ms). Terefore, it can rocess /29.6 = 33 CIF frames er second. FPGA resource usages of bot DBF imlementations including on ci memories are sown in Table 2.2. LUMA SRAM and CHROMA SRAM are imlemented as dualort block SelectRAMs. SPAD and IBUF are imlemented as dual-ort distributed SelectRAMs. Bot DBF ardware imlementations are syntesized to.8 µm UMC standard cell library. Bot ardware imlementations can work at 2 MHz and tey can rocess 3 VGA (64x48) frames er second. DBF_4 4 and DBF_6 6 ardware imlementations, excluding on-ci memories, are syntesized to 7.4 K and 5.3 K gates resectiely. As sown in Table 2.3, tese gate counts are te lowest among te H.264 DBF ardware imlementations resented in te literature [6,3,3,32,33]. Tese ardware imlementations aciee ig erformance at te exense of ig ardware cost. We 23

39 acieed real-time erformance by only using one 2-bit adder, one 2-bit comarator, a few sifters, and a number of multilexers in our dataat. Table 2.2 FPGA Resource Usage and Clock Frequency After Place and Route Resource DBF_4x4 Hardware DBF_6x6 Hardware Function Generators DFFs Block SelectRAMs 2 2 Clock Frequency 67 MHz 72 MHz Table 2.3 DBF Hardware Comarison Category [7] [] [2] [3] [32] DBF_6x6 Gate Count 2.6 K 9.2 K.8 K 4.8 K 7.5 K 5.3 K Tecnology.25 µ.8 µ.8 µ.8 µ.3 µ.8 µ UMC Artisan UMC UMC UMC TSMC On-ci Memory Size 6x32 8x32 4x32 6x32 32x32 384x8 2.4 ARM Versatile / PB926EJ-S Deeloment Board Imlementation Bot DBF ardware imlementations are erified to work correctly in te ARM Versatile PB926EJ-S deeloment enironment sown in Figure 2.9. As sown in te figure, te deeloment enironment consists of a PC connected to ARM Versatile PB926EJ-S board troug ARM Multi-ICE, a logic tile mounted on te Versatile PB926EJ-S baseboard and a color LCD anel [34]. PC is used to create te bit stream tat will be loaded into te 8-million-gate Xilinx Virtex II FPGA on te logic tile wic can be configured to imlement custom-designed logic. ARM Multi-ICE is used for communicating between PC and Arm Versatile board, and AXD Debugger from ARM Deeloer Suite is used for debugging te system. Te Color LCD anel is used to dislay images for isual erification. 24

40 Figure 2.9 ARM Versatile / PB926EJ-S Deeloment Enironment and Power Measurement Setu As sown in Figure 2., an AHB bus interface is designed and integrated into DBF ardware in order to communicate wit ARM rocessor and external SRAM troug AHB bus, and DBF Hardware is integrated into te FPGA on te logic tile as a master of te AHB S bus. A ideo frame is loaded into SRAM located on te board from PC using software. Tis ideo frame is used as an inut to DBF ardware running on te FPGA. DBF ardware alies te H.264 DBF algoritm to tis ideo frame and writes te resulting frame back to SRAM. Te resulting ideo frame is sown on te color LCD anel. An unfiltered ideo frame and te same ideo frame filtered by H.264 DBF ardware running in te FPGA on te logic tile are sown in Figure 2. and Figure 2.2. As it can be seen from te figure, some of te blocking artifacts in te unfiltered ideo frame are reduced and some of tem are totally remoed. 25

41 Figure 2. Integration of Deblocking Filter Hardware into ARM Versatile Board 2.5 Power Consumtion Results Te ower consumtions of bot DBF ardware imlementations on a Xilinx Virtex II FPGA are estimated using Xilinx XPower tool. In order to estimate te ower consumtion of a DBF ardware imlementation, timing simulation of te laced and routed netlist of tat DBF ardware imlementation is done using Mentor Graics ModelSim SE for one frame of Foreman ideo sequence and te signal actiities are stored in a VCD file. Tis VCD file is used for estimating te ower consumtion of tat DBF ardware imlementation using Xilinx XPower tool. Te ower consumtions of bot DBF ardware imlementations on a Xilinx Virtex II FPGA at 5 MHz are sown in Table 2.4. Since DBF ardware will be used as art of an H.264 encoder or decoder only internal ower consumtion is considered and inut and outut ower consumtions are ignored. To make a fair comarison between te ower consumtions of te two DBF ardware imlementations, we used same number of distributed SelectRAMs and block SelectRAMs for bot imlementations. As sown in te table, DBF_6x6 ardware as 36% less ower consumtion tan DBF_4x4 ardware. 26

42 Figure 2.Unfiltered Video Frame Figure 2.2 Te Same Frame Filtered by H.264 Deblocking Filter Algoritm 27

43 Table 2.4 Power Consumtion of DBF Hardware Imlementations at 5 MHz Category DBF_4 4 DBF_6 6 Clock mw 5.36 mw Logic mw mw Signal mw mw Total mw mw Table 2.5 Power Consumtion Comarison of Block SelectRAM and Distributed Category SelectRAM Maximum Switcing Actiity Minimum Switcing Actiity Block SelectRAM 2.2 mw 3.7 mw Distributed SelectRAM mw 2.67 mw Te ower consumtion of a DBF ardware imlementation can be diided into tree main categories; signal ower, logic ower and clock ower. Signal ower is te ower dissiated in routing tracks between logic blocks. A significant amount of ower is dissiated in routing tracks. It accounts for 29% of total ower consumtion of DBF_4 4 ardware and 43% of total ower consumtion of DBF_6 6 ardware. Logic ower is te amount of ower dissiated in te arts were comutations take lace. Clock ower is due to clock tree used in te FPGA. Since tere is less number of fli-flos in DBF_6 6 ardware in comarison wit te DBF_4 4 ardware, te clock ower of DBF_6x6 ardware is less tan te clock ower of DBF_4x4 ardware. Xilinx Virtex-II FPGAs ae block SelectRAM and distributed SelectRAM memories. In DBF ardware imlementations, we used bot block SelectRAMs and distributed SelectRAMs as local memories for storing intermediate results. We, terefore, caracterized te ower consumtions of block SelectRAMs and distributed SelectRAMs using Xilinx XPower tool for te cases wen tere is maximum switcing actiity and minimum switcing actiity in te RAMs, and te results are sown in Table 2.5. Te results sow tat te ower consumtion of a distributed SelectRAM is muc more tan te ower consumtion of a block SelectRAM. Tis is because a distributed 28

44 SelectRAM is formed by look u tables in Configurable Logic Blocks (CLBs) and tis causes te memory to be distributed in te FPGA and ae long interconnects. On te oter and, a block SelectRAM is a carefully designed and otimized full-custom SRAM. Terefore, we decided to use only block SelectRAMs in DBF_6 6 ardware. Using block SelectRAMs instead of distributed SelectRAMs in DBF_6 6 ardware roided additional 28% ower reduction and total ower consumtion of DBF_6x6 ardware is reduced from mw to 3.45 mw. In addition, we alied clock gating and glitc reduction tecniques to DBF dataat for reducing its ower consumtion. DBF dataat is two-stage ielined. Te first stage erforms numerical calculations eery clock cycle, but te second stage is not actie for a considerable amount of clock cycles. Terefore, we turned off te second stage by clock gating wen it is inactie. Table 2.6 sows te imact of clock gating on dataat ower consumtion. Te dataat ower consumtion is reduced by 3% using clock gating. Table 2.6 Imact of Clock Gating on Dataat Power Consumtion Category Dataat Dataat wit Clock Gating Clock 7.46 mw 6.7 mw Logic 7.62 mw 5.88 mw Signal 8.4 mw 6.56 mw Total mw 29.5 mw Table 2.7 Imact of Glitc Reduction on Dataat Power Consumtion Category Dataat Dataat witout Glitces Pielined Dataat Clock 7.46 mw 7.37 mw 9,25 mw Logic 7.62 mw 6.6 mw 6,7 mw Signal 8.4 mw 6.47 mw 6,59 mw Total mw 3.44 mw 3,9 mw Glitc is a surious transition at a node witin a single cycle before te node settles to te correct logic alue. Unlike ASICs, in wic signals can be routed using any 29

45 aailable silicon, FPGAs imlement interconnects using fixed metal tracks and rogrammable switces. Te relatie scarcity of rogrammable switces often forces signals to take longer routes tan would be seen in an ASIC. As a result, te otential for unequal delays among signals, and ence te creation of glitces, is more likely tan tat in an ASIC. Tus, reducing glitces by ielining is an effectie ower reduction tecnique for FPGAs [2]. Te imact of glitces on DBF dataat ower consumtion can be seen by simulating te dataat under zero delay model and analyzing its ower consumtion. Te glitc free ower consumtion of DBF dataat is sown in Table 2.7. Te glitc free ower consumtion sows te maximum ower consumtion reduction tat can be obtained by reducing glitces. Table 2.7 sows te imact of reducing glitces by ielining on dataat ower consumtion. We inserted two ieline registers immediately before te inuts of te adder. Tis reduced te dataat ower consumtion by 4.7%. We terefore obtained 5% of maximum ossible ower reduction tat can be obtained by reducing glitces. We also measured te ower consumtions of bot DBF ardware imlementations on a Xilinx Virtex II FPGA using te setu sown in Figure 2.9. Using tis setu, we measured te aerage current before DBF ardware is running on te FPGA. We, ten, measured te aerage current wile DBF ardware is running on te FPGA at 34 MHz in a continuous loo. Since te FPGA on te logic tile is sulied wit 3.3 V ower suly, te ower consumtion of DBF ardware is calculated by multilying te difference in aerage current wit 3.3 V. Te ower consumtion measurement and estimation results are sown in Table 2.8. DBF_4x4 ardware used for tese measurements and estimations as 3 distributed SelectRAMs and 2 block SelectRAMs, oweer, DBF_6 6 ardware used for tese measurements and estimations as 5 block SelectRAMs. Te ower consumtion measurement results are sligtly larger tan te ower consumtion estimation results. Te difference between measured and estimated results is caused by te ower consumed for reading te unfiltered MBs from and writing te filtered MBs to te SRAM on te logic tile troug AHB bus wic is not included in ower consumtion estimations. 3

46 Table 2.8 Power Consumtion Estimations and Measurements of DBF_6 6 and DBF Hardware DBF_4 4 Hardware at 34 MHz Aerage Current witout DBF Aerage Current wit DBF Estimated Power Measured Power DBF_ ma 76 ma 22.6 mw 254. mw DBF_6 6 9 ma 52 ma 89.7 mw 8.9 mw 2.6 A Noel Comutational Comlexity Reduction Tecnique for H.264 Deblocking Filter In tis section, we roose a noel comutational comlexity reduction tecnique wic aoids unnecessary calculations in H.264 DBF and terefore reduces te ower consumtion of H.264 DBF ardware. H.264 DBF algoritm is igly adatie and eigt different filtering equations are used based on BS, α and β arameters and ixel gradient as sown in Figure 2.3. Eigt ossible conditions, called modes, and te ixels used in filtering equations used in tese modes are listed in Table 2.9. Te filtering equations used in eac mode are gien in Figure 2.3. As it can be seen from filtering equations, H.264 DBF algoritm can be imlemented using only addition and sift oerations. If some or all of te ixels used in filtering equations are equal, ten te filtering equations eiter simlify significantly or become unnecessary. Te filtering equations used in mode 6 are gien in Table 2.. For mode 6, if te ixels tat will be filtered are all equal,, and q become zero and filtering becomes unnecessary. In addition, if some of te ixels tat will be filtered are equal, te filtering equations used in tis mode simlify significantly. Table 2. sows suc a case in wic ==q=q=. As sown in Tables 2. and 2., some or all of te calculations are aoided if te equality of te ixels tat will be filtered is known before te filtering rocess. 3

47 Table 2.9 DBF Modes BS 2- <β q2-q <β Pixels used in filtering equations Mode BS=4 4>BS> False False,, q, q False True,, q, q, q2, q3 3 True False 3, 2,,, q, q, 5 True True 3, 2,,, q, q, q2, q3 7 False False,, q, q False True,, q, q, q2 2 True False 2,,, q, q 4 True True 2,,, q, q, q2 6 Table 2. Equations for Mode 6 and teir Simlified Versions wen 2===q=q=q2 Equations for mode 6 Simlified equations for mode 6 wen 2===q=q=q2 ' = ' = q' = q- q' = q ' = ' = q' = q q q' = q i = (4(q-)(-q)4)>>3 i = = Min(Max(-c, i),c) = i = (2((q)>>)-2)>> i = = Min(Max(-c, i),c) = qi = (q2((q)>>)-2q)>> qi = q = Min(Max(-c, qi),c) q = Table 2. Equations for Mode 6 and teir Simlified Versions wen ==q=q Equations for mode 6 Simlified equations for mode 6 wen ==q=q ' = ' = q' = q- q' = q ' = ' = q' = q q q' = q q i = (4(q-)(-q)4)>>3 i = = Min(Max(-c, i),c) = i = (2((q)>>)-2)>> i = (2-) >> = Min(Max(-c, i),c) = Min(Max(-c, i),c) qi = (q2((q)>>)-2q)>> qi = (q2-) >> q = Min(Max(-c, qi),c) q = Min(Max(-c, qi),c) 32

48 In order to aoid te oeread for determining te equality of te ixels tat will be filtered, we roose to use te subtraction oerations erformed in conditional brances of DBF algoritm as sown in Figure 2.3. Tese conditional brances include te fie subtraction oerations sown in (2.)-(2.5). If two ixels are equal, teir difference will be equal to zero. Terefore, by only cecking te results of tese fie subtraction oerations, we can determine te equality of te ixels tat will be filtered witout erforming additional comarison oerations. For examle, if te results of all te equations sown in (2.)-(2.5) are zero, following ixels become equal 2===q=q=q2. If equations (2.), (2.2), and (2.4) are zero, ten following ixels become equal 2===q. q (2.) (2.2) q q (2.3) 2 (2.4) q2 q (2.5) Te amount of simlification in te filtering equations deends on te DBF mode and te ixels tat are equal. We calculated bot te amount of addition and sift oerations erformed for filtering in eac mode, and te amount of addition and sift oerations erformed for filtering for eac mode wen different combinations of ixels tat will be filtered are equal. Te results are sown in Tables In tese tables, CB in category column denotes te subtraction oerations erformed before te filtering. 33

49 Table 2.2 Te Amount of Comutation Required by DBF Mode For Different Equal Pixel Combinations Category Original If =q If = =q =q # of Add. # of Sifts # of Add. # of Sifts # of Add. # of Sifts CB oi ', q' 2 2 Total Table 2.3 Te Amount of Comutation Required by DBF Mode For Different Equal Pixel Combinations Category Original If = =q or =q =q # of # of Add. Sifts If = =q =q # of Add. # of Sifts # of Add. CB ', q' Total # of Sifts Table 2.4 Te Amount of Comutation Required by DBF Mode 2 For Different Equal Pixel Combinations Category Original If =q If =q =q If = = q =q # of # of Add. Sifts If = =q =q =q 2 # of # of Add. Sifts # of Add. # of Sifts # of Add. # of Sifts # of Add. # of Sifts CB oi qi ', q', q' c Total

50 Table 2.5 Te Amount of Comutation Required by DBF Mode 3 For Different Equal Pixel Combinations Category Original If =q If =q =q If = =q =q If = =q =q =q 2 # of # of Add. Sifts If = = q =q =q 2 =q 3 # of # of Add. Sifts # of Add. # of Sifts # of Add. # of Sifts # of Add. # of Sifts # of Add. # of Sifts CB ' q' q' q' (α>>2)2 Total Table 2.6 Te Amount of Comutation Required by DBF Mode 5 For Different Equal Pixel Combinations Category Original If =q If = =q If = =q =q # of # of Add. Sifts If 2 = = =q =q # of # of Add. Sifts If 3 = 2 = = =q =q # of # of Add. Sifts # of Add. # of Sifts # of Add. # of Sifts # of Add. # of Sifts CB ' ' ' q' (α>>2)2 Total

51 Table 2.7 Te Amount of Comutation Required by DBF Mode 6 For Different Equal Pixel Combinations Original =q = =q =q =q = = 2 = = = = =q = 2 = = = q =q q =q q =q 2 q =q =q 2 Category A S A S A S A S A S A S A S A S CB oi i qi ', q', ', q' c Total Category Table 2.8 Te Amount of Comutation Required by DBF Mode 7 For Different Equal Pixel Combinations = =q / Original =q =q =q / 2 = = = = =q = 2 = = = 3 = 2 = = 2 = = = q = =q =q =q q =q 2 q =q =q =q =q = q =q =q 2 = 2 q 2 q 3 3 = 2 = = =q =q = q 2 =q 3 A S A S A S A S A S A S A S A S A S CB ' ' ' q' q' q' (α>>2)2 Total

52 Table 2.9 Te Amount of Comutation Required by DBF Mode 4 For Different Equal Pixel Combinations Category Original If =q If = =q If = =q =q # of # of Add. Sifts If 2 = = =q =q # of # of Add. Sifts # of Add. # of Sifts # of Add. # of Sifts # of Add. # of Sifts CB oi i ', q', ' c Total Te number of addition and sift oerations required by eac DBF mode can be obtained from te filtering equations sown in Figure 2.3. But, since modes 3, 5, and 7 ae common arts in filtering equations, te results of one equation can be reused for oter equations. Terefore, we organized te filtering equations used in modes 3, 5 and 7 using data reuse in order to reduce te amount of comutations. Te filtering equations used in mode 7 are sown in (2.6)-(2.). Table 2.2 sows te organized filtering equations and te number of addition and sift oerations required by tese equations. Te filtering equations used in modes 3 and 5 are also organized using data reuse. Te number of occurrences of different equal ixel combinations for DBF mode 6 obtained by using H.264 JM reference software ersion 4. for seeral ideo sequences are sown in Table 2.2. In tis table, te column Total sows te number of all occurrences of DBF mode 6. In te simulations, eigt CIF size ideos are used and 3 frames are encoded for eac ideo. I-frame eriod is set to frames (IPPPPPPPPPIP ). By using te subtraction oerations sown in (2.)-(2.5), we can only ceck te equality of tree ixels (2 - q2) on eac side of an edge. Howeer, for modes 3, 5, and 7, wen BS = 4, DBF algoritm can access u to four ixels on eac side of an edge, and terefore 3 or q3 or bot can be used in filtering calculations. Extra comarison oerations can be erformed to determine te equality of ixels 3/q3 wit te oter ixels tat will be filtered for modes 3, 5, and 7. Howeer, te simulation results obtained 37

53 by H.264 JM reference software ersion 4. for seeral ideo sequences sowed tat te number of occurrences of mode 3 wit ==q=q=q2=q3 and mode 5 wit 3=2===q=q is ery low. Te occurrence of mode 7 wit 3=2===q=q=q2=q3 is quite ig. Terefore, we roose tat comarison of 3/q3 wit te oter ixels tat will be filtered is erformed only for mode 7. We determined te comutation reduction acieed by te roosed tecnique for H.264 DBF algoritm using H.264 JM reference software ersion 4. and te information gien in Tables As sown in Table 2.22, te amount of reductions acieed in addition and sift oerations ranges from 6% to 39% and 8% to 5% resectiely. Te roosed tecnique, on te oter and, as to ceck if te results of seeral subtraction oerations are equal to zero or not. But tis oeread is quite small considering tat cecking weter a number is zero or not can be efficiently imlemented in ardware. '=(2222qq4)>>3 (2.6) '=(2q2)>>2 (2.7) '2=(2332q4)>>3 (2.8) q'=(q22q2q24)>>3 (2.9) q'=(q2qq2)>>2 (2.) q'2=(2q33q2qq4)>>3 (2.) Table 2.2 Te Amount of Comutation Required by DBF Mode 7 Equations Number of Number of Additions Sifts A = q2 2 '=(2A)>>2 2 '=( 2-2q)>>3 2 2 '2=(2(32) 2)>>3 3 2 q'=(q2qa)>>2 2 q =( 2q -q2)>>3 2 2 q 2=(2(q3q2)q 2)>>3 3 2 Total 6 38

54 Table 2.2 Number of Occurrences of Different Equal Pixel Combinations for DBF Mode 6 Video Foreman Akiyo Moter Daugter Football Bus Mobile Soccer Ice QP 2===q =q=q2 2===q=q or ==q=q=q2 ==q=q ==q or =q=q =q Number Percent Number Percent Number Percent Number Percent Number Percent % % % % % % % % % % % % 26.5% % % % % 488.5% % 53.57% % % 38.3% % 39.37% % % % % 79.7% % % % % % % % % % % % % % % 48.7% % % 5.37% % % % % 57.4% % % % % 836.4% % % % % % % % % % % % % % % 869.4% % 4.83% % % % % % % % 678.6% % % % % 424.4% % % % % % % % % % 77.5% % % % % 45.5% % % % % 643.7% % % % % 22.58% % 362.7% % % % % % Total 39

55 Table 2.23 sows te number of comarisons of 3/q3 wit te oter ixels tat will be filtered, te amount of addition reductions acieed by te roosed tecnique, and te ercentage of comarison oeread to te amount of addition reductions. As it can be seen from te table, te oeread of comaring 3/q3 wit te oter ixels tat will be filtered is muc smaller tan te amount of addition reductions acieed by te roosed tecnique. Table 2.22 Comutation Reduction Results Video Foreman Akiyo Moter Daugter Football Bus Mobile Soccer Ice QP Addition Sift Total Reduc. Percent Total Reduc. Percent % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % 4

56 Table 2.23 Comarison Oeread Video Foreman Akiyo Moter Daugter Football Bus Mobile Soccer Ice QP Comarison Addition Oeread Reduction Percent % % % % % % % % % % % % % % % % % % % % % % % % 4

57 3 CHAPTER III LOW POWER H.264 INTRA PREDICTION HARDWARE DESIGNS H.264 intra rediction algoritm as a ery ig comutational comlexity. In tis cater, we roose a noel tecnique for reducing te amount of comutations erformed by H.264 intra rediction algoritm and terefore reducing te ower consumtion of H.264 intra rediction ardware significantly witout any PSNR and bit rate loss. Te roosed tecnique erforms a small number of comarisons among neigboring ixels of te current block before te intra rediction rocess. If te neigboring ixels of te current block are equal, te rediction equations of H.264 intra rediction modes simlify significantly for tis block. By exloiting te equality of te neigboring ixels, te roosed tecnique reduces te amount of comutations erformed by 4x4 luminance, 6x6 luminance, and 8x8 crominance rediction modes u to 6%, 28%, and 68% resectiely wit a small comarison oeread. We also imlemented an efficient 4x4, 6x6 and 8x8 intra rediction ardware arcitectures including te roosed tecnique using Verilog HDL. We quantified te imact of te roosed tecnique on te ower consumtion of tese ardware arcitectures on a Xilinx Virtex II FPGA using Xilinx XPower, and it reduced te ower consumtion of 4x4, 6x6 and 8x8 ardware arcitectures u to 8.6%, 8.3% and 2.5% resectiely. 42

58 Te roosed tecnique is alicable to H.264 4x4 luminance, 6x6 luminance and 8x8 crominance rediction modes. Te tecnique erforms a small number of comarisons among neigboring ixels of te current block before te intra rediction rocess. If te neigboring ixels used for calculating te redicted ixels by an intra 4x4 rediction mode are all equal, te redicted ixels by tis mode also become equal. Terefore, te rediction equations simlify to a constant alue and rediction calculations for tis mode become unnecessary. Furtermore, if te neigboring ixels used for calculating te redicted ixels by an intra 6x6 or an intra 8x8 rediction mode are equal, te rediction equations used by tis mode simlify significantly. Te simulation results obtained by H.264 reference software, JM 4. [35], for seeral ideo sequences sowed tat tis tecnique reduces te amount of comutations erformed by H.264 intra 4x4, 6x6 and 8x8 rediction modes u to 6%, 28%, and 68% resectiely wit a small comarison oeread. Te roosed tecnique, for eac MB, requires 2 and 24 comarisons for intra 4x4 and intra 8x8 rediction modes resectiely. Since intra 4x4 and intra 6x6 rediction modes oerate on te same MB, te comarison results for intra 4x4 rediction modes are also used for intra 6x6 rediction modes. Seeral tecniques are reorted in te literature for reducing te comutational comlexity of H.264 intra rediction algoritm [36,37,38,39]. Tese tecniques reduce te amount of comutation for H.264 intra rediction algoritm by trying selected intra rediction modes rater tan trying all intra rediction modes. Howeer, tese tecniques ae a drawback of PSNR and bit rate loss and tey require significant amount of recomutation. We also designed efficient H.264 4x4, 6x6 and 8x8 intra rediction ardware arcitectures including te roosed tecnique. Te ardware arcitectures are imlemented in Verilog HDL. Te Verilog RTL codes are erified to work at 5 MHz in a Xilinx Virtex II FPGA. Te imact of tis tecnique on te ower consumtion of tese ardware imlementations on a Xilinx Virtex II FPGA is quantified using Xilinx XPower tool. Te roosed tecnique reduced te ower consumtion of 4x4, 6x6 and 8x8 ardware arcitectures u to 8.6%, 8.3% and 2.5% resectiely. 43

59 Seeral ardware arcitectures for H.264 4x4 intra rediction algoritm are reorted in te literature [4,4,42,43]. Howeer, tey do not reort teir ower consumtion and tey do not imlement te roosed tecnique. 3. H.264 Intra Prediction Algoritm Oeriew Intra rediction algoritm redicts te ixels in a MB using te ixels in te aailable neigboring blocks. For te luma comonent of a MB, a 6x6 redicted luma block is formed by erforming intra redictions for eac 4x4 luma block in te MB and by erforming intra rediction for te 6x6 MB. Tere are nine rediction modes for eac 4x4 luma block and four rediction modes for a 6x6 luma block. A mode decision algoritm is ten used to comare te 4x4 and 6x6 redictions and select te best luma rediction mode for te MB. 4x4 rediction modes are generally selected for igly textured regions wile 6x6 rediction modes are selected for flat regions. Tere are nine 4x4 luma rediction modes designed in a directional manner. A 4x4 luma block consisting of te ixels a to is sown in Figure 3.. Te ixels A to M belong to te neigboring blocks and are assumed to be already encoded and reconstructed and are terefore aailable in te encoder and decoder to generate a rediction for te current MB. Eac 4x4 luma rediction mode generates 6 redicted ixel alues using some or all of te neigboring ixels A to M as sown in Figure 3.2. Te examles of eac 4x4 luma rediction mode for real images are gien in Figure 3.3. Te arrows indicate te direction of rediction in eac mode. Te redicted ixels are calculated by a weigted aerage of te neigboring ixels A-M for eac mode excet Vertical, Horizontal and DC modes. Te rediction equations used in eac 4x4 luma rediction mode are sown in Figure 3.4 were [x, y] denotes te osition of te ixel in a 4x4 block (te to left, to rigt, bottom left, and bottom rigt ositions of a 4x4 block are denoted as [, ], [, 3], [3, ], and [3, 3], resectiely) and red[x, y] is te rediction for te ixel in te osition [x, y]. 44

60 Figure 3. A 4x4 Luma Block and Neigboring Pixels DC mode is always used regardless of te aailability of te neigboring ixels. Howeer, it is adoted based on wic neigboring ixels A-M are aailable. If ixels E, F, G and H ae not yet been encoded and reconstructed, te alue of ixel D is coied to tese ositions and tey are marked as aailable for DC mode. Te oter rediction modes can only be used if all of te required neigboring ixels are aailable [,3]. Aailable 4x4 luma rediction modes for a 4x4 luma block deending on te aailability of te neigboring 4x4 luma blocks are gien in Table 3.. Figure 3.2 4x4 Luma Prediction Modes 45

61 Figure 3.3 Examles of Real Images for 4x4 Luma Prediction Modes red[, ] = A red[, ] = B red[, 2] = C red[, 3] = D red[, ] = A red[, ] = B red[, 2] = C red[, 3] = D red[2, ] = A red[2, ] = B red[2, 2] = C red[2, 3] = D red[3, ] = A red[3, ] = B red[3, 2] = C red[3, 3] = D (a) 4x4 Vertical Mode red[, ] = I red[, ] = I red[, 2] = I red[, 3] = I red[, ] = J red[, ] = J red[, 2] = J red[, 3] = J red[2, ] = K red[2, ] = K red[2, 2] = K red[2, 3] = K red[3, ] = L red[3, ] = L red[3, 2] = L red[3, 3] = L (b) 4x4 Horizontal Mode 46

62 red[x, y] = (A B C D I J K L 4) >> 3 red[x, y] = (I J K L 2) >> 2 red[x, y] = (A B C D 2) >> 2 red[x, y] = 28 (c) 4x4 DC Mode If te left and te to neigboring ixels are aailable Else If only te left neigboring ixels are aailable Else If only te to neigboring ixels are aailable Else red[, ] = A 2B C 2 >> 2 red[, ] = A 2M I 2 >> 2 red[, ] = B 2C D 2 >> 2 red[, ] = M 2A B 2 >> 2 red[, 2] = C 2D E 2 >> 2 red[, 2] = A 2B C 2 >> 2 red[, 3] = D 2E F 2 >> 2 red[, 3] = B 2C D 2 >> 2 red[, ] = B 2C D 2 >> 2 red[, ] = M 2I J 2 >> 2 red[, ] = C 2D E 2 >> 2 red[, ] = A 2M I 2 >> 2 red[, 2] = D 2E F 2 >> 2 red[, 2] = M 2A B 2 >> 2 red[, 3] = E 2F G 2 >> 2 red[, 3] = A 2B C 2 >> 2 red[2, ] = C 2D E 2 >> 2 red[2, ] = I 2J K 2 >> 2 red[2, ] = D 2E F 2 >> 2 red[2, ] = M 2I J 2 >> 2 red[2, 2] = E 2F G 2 >> 2 red[2, 2] = A 2M I 2 >> 2 red[2, 3] = F 2G H 2 >> 2 red[2, 3] = M 2A B 2 >> 2 red[3, ] = D 2E F 2 >> 2 red[3, ] = J 2K L 2 >> 2 red[3, ] = E 2F G 2 >> 2 red[3, ] = I 2J K 2 >> 2 red[3, 2] = F 2G H 2 >> 2 red[3, 2] = M 2I J 2 >> 2 red[3, 3] = G 3H 2 >> 2 red[3, 3] = A 2M I 2 >> 2 (d) 4x4 Diagonal Down Left Mode (e) 4x4 Diagonal Down Rigt Mode red[, ] = M A >> red[, ] = M I >> red[, ] = A B >> red[, ] = I 2M A 2 >> 2 red[, 2] = B C >> red[, 2] = B 2A M 2 >> 2 red[, 3] = C D >> red[, 3] = C 2B A 2 >> 2 red[, ] = I 2M A 2 >> 2 red[, ] = I J >> red[, ] = M 2A B 2 >> 2 red[, ] = M 2I J 2 >> 2 red[, 2] = A 2B C 2 >> 2 red[, 2] = M I >> red[, 3] = B 2C D 2 >> 2 red[, 3] = I 2M A 2 >> 2 red[2, ] = M 2I J 2 >> 2 red[2, ] = J K >> red[2, ] = M A >> red[2, ] = I 2J K 2 >> 2 red[2, 2] = A B >> red[2, 2] = I J >> red[2, 3] = B C >> red[2, 3] = M 2I J 2 >> 2 red[3, ] = I 2J K 2 >> 2 red[3, ] = K L >> red[3, ] = I 2M A 2 >> 2 red[3, ] = J 2K L 2 >> 2 red[3, 2] = M 2A B 2 >> 2 red[3, 2] = J K >> red[3, 3] = A 2B C 2 >> 2 red[3, 3] = I 2J K 2 >> 2 (f) 4x4 Vertical Rigt Mode (g) 4x4 Horizontal Down Mode 47

63 red[, ] = A B >> red[, ] = I J >> red[, ] = B C >> red[, ] = I 2J K 2 >> 2 red[, 2] = C D >> red[, 2] = J K >> red[, 3] = D E >> red[, 3] = J 2K L 2 >> 2 red[, ] = A 2B C 2 >> 2 red[, ] = J K >> red[, ] = B 2C D 2 >> 2 red[, ] = J 2K L 2 >> 2 red[, 2] = C 2D E 2 >> 2 red[, 2] = K L >> red[, 3] = D 2E F 2 >> 2 red[, 3] = K 3L 2 >> 2 red[2, ] = B C >> red[2, ] = K L >> red[2, ] = C D >> red[2, ] = K 3L 2 >> 2 red[2, 2] = D E >> red[2, 2] = L red[2, 3] = E F >> red[2, 3] = L red[3, ] = B 2C D 2 >> 2 red[3, ] = L red[3, ] = C 2D E 2 >> 2 red[3, ] = L red[3, 2] = D 2E F 2 >> 2 red[3, 2] = L red[3, 3] = E 2F G 2 >> 2 red[3, 3] = L () 4x4 Vertical Left Mode (i) 4x4 Horizontal U Mode Figure 3.4 Prediction Equations for 4x4 Luma Prediction Modes Table 3. Aailability of 4x4 Luma Prediction Modes Aailability of Neigboring 4x4 Luma Blocks None aailable Left aailable, To not aailable To aailable, Left not aailable Bot aailable Aailable 4x4 Luma Prediction Modes DC Horizontal, DC, Horizontal U Vertical, DC, Vertical Left, Diagonal Down-Left All Modes Tere are four 6x6 luma rediction modes designed in a directional manner. Eac 6x6 luma rediction mode generates 256 redicted ixel alues using some or all of te uer (H) and left-and (V) neigboring ixels as sown in Figure 3.5. Vertical, Horizontal and DC modes are similar to 4x4 luma rediction modes. Plane mode is an aroximation of bilinear transform wit only integer aritmetic. Te examles of eac 6x6 luma rediction mode for real images are gien in Figure 3.6. Te rediction equations used in 6x6 luma rediction modes are sown in Figure 3.7 were [y, x] denotes te osition of te ixel in a MB (te to left, to rigt, bottom left, and bottom 48

64 rigt ositions of a MB are denoted as [,], [,5], [5,], and [5,5], resectiely), reresents te neigboring ixel alues and Cli is to cli te result between and 255. DC mode is always used regardless of te aailability of te neigboring ixels. Howeer, it is adoted based on wic neigboring ixels are aailable. Te oter rediction modes can only be used if all of te required neigboring ixels are aailable [,3]. Aailable 6x6 luma rediction modes for a MB deending on te aailability of te neigboring MBs are gien in Table 3.2. Figure 3.5 6x6 Luma Prediction Modes Figure 3.6 Examles of Real Images for 6x6 Luma Prediction Modes 49