libaom 源码分析：运动搜索过程和 pattern_search 函数

AV1AV1AOMedia Video 1是一种开源、免专利费的视频编码格式旨在提高视频压缩效率和传输效果。它由开放媒体联盟AOMedia开发该联盟包括谷歌、微软、亚马逊、苹果和Netflix等大型科技公司。AV1特别适合用于互联网视频流媒体传输因为它相较于旧格式如H.264或VP9提供了更高的压缩效率可以在更低的比特率下提供更高质量的视频。AV1的主要特点更优的压缩效率AV1在压缩效率上比H.264高达50%比VP9高约30%。免专利费与H.265/HEVC不同AV1无需支付许可费用适合开发者和平台使用。开源其规范和实现公开促进了广泛的采用。广泛支持许多现代浏览器、流媒体平台如YouTube、Netflix和硬件设备都已支持AV1。面向未来AV1专为高分辨率内容如4K、8K、HDR设计具有强大的未来适应性。应用领域流媒体传输YouTube和Netflix等平台使用AV1进行高质量视频流媒体传输。视频会议由于带宽需求较低AV1非常适合用于实时视频通话。云游戏AV1可通过提供高质量图像并降低延迟优化云游戏体验。如果您想了解更多关于AV1的技术细节、编码工具或具体应用请告诉我libaomLibaom指的是开源的 AV1 视频编码库和实现。AV1 是一种高效的视频编码格式由开放媒体联盟AOMedia开发旨在提供比以前的编码格式如 VP9、HEVCH.265和 AVCH.264更高的压缩效率。它特别优化用于流媒体、网页视频、视频会议等应用。Libaom是 AV1 的参考编码器和解码器由 AOMedia 项目开发和维护。它被内容提供商和开发人员广泛使用特别是那些需要将 AV1 视频编码和解码集成到其软件中的开发者尤其是寻求通过降低带宽要求来提供高质量视频的流媒体平台。Libaom 和 AV1 的一些特点包括更好的压缩AV1 通常比 HEVCH.265提供约 30% 更好的压缩效率在相同质量下可以减少带宽消耗这使其成为降低数据使用量的有吸引力的选择。免版权费与 HEVC 不同AV1 设计为免版权费促进了它在各种应用中的采用。开源Libaom 库是开源的这意味着开发者可以自由访问和修改代码以满足他们的需求。Libaom 主要用于软件编码场景由于其更复杂的算法它的性能可能比某些硬件编码器慢。然而凭借其高效性和潜在的成本节省它在视频流媒体服务中越来越受欢迎。libaom 运动搜索过程流程框图主要搜索方法FAST_BIGDIA、VFAST_DIAMOND、FAST_DIAMOND、FAST_HEX、HEX、SQUARE、BIGDIA快速搜索方法与普通搜索方法区别从源码调用看区别在是否开启了初始搜索即在快搜索类中关闭了 do_init_search 开关而在普通搜索里开启了do_init_search 开关核心函数patter_search 函数根据不同的模版和条件进行不同的像素搜索计算代价选择最合适的候选点。pattern_search 源码详细分析// Generic pattern search function that searches over multiple scales.// Each scale can have a different number of candidates and shape of// candidates as indicated in the num_candidates and candidates arrays// passed into this function//用于在不同的搜索尺度和候选点形状下搜索最佳整数像素运动矢量Motion VectorMV// best_mv: 返回的最佳运动矢量// best_mv_stats: 返回的最佳矢量统计信息staticintpattern_search(FULLPEL_MV start_mv,constFULLPEL_MOTION_SEARCH_PARAMS*ms_params,intsearch_step,constintdo_init_search,int*cost_list,FULLPEL_MV*best_mv,FULLPEL_MV_STATS*best_mv_stats){//搜索步长集合staticconstintsearch_steps[MAX_MVSEARCH_STEPS]{10,9,8,7,6,5,4,3,2,1,0,};inti,s,t;conststructbuf_2d*constsrcms_params-ms_buffers.src;conststructbuf_2d*constrefms_params-ms_buffers.ref;constsearch_site_config*search_sitesms_params-search_sites;constint*num_candidatessearch_sites-searches_per_step;constintref_strideref-stride;constintlast_is_4num_candidates[0]4;intbr,bc;unsignedintbestsadUINT_MAX,raw_bestsadUINT_MAX;intk-1;constMV_COST_PARAMS*mv_cost_paramsms_params-mv_cost_params;search_stepAOMMIN(search_step,MAX_MVSEARCH_STEPS-1);//获取初始搜索步长assert(search_step0);intbest_init_ssearch_steps[search_step];// adjust ref_mv to make sure it is within MV range 确保初始 mv 再 mv 限制范围内clamp_fullmv(start_mv,ms_params-mv_limits);brstart_mv.row;bcstart_mv.col;//初始化成本列表if(cost_list!NULL){cost_list[0]cost_list[1]cost_list[2]cost_list[3]cost_list[4]INT_MAX;}intcostlist_has_sad0;// Work out the start point for the search// 计算初始 mv 的 SAD 和误差代价作为当前最优值的起点raw_bestsadget_mvpred_sad(ms_params,src,get_buf_from_fullmv(ref,start_mv),ref_stride);bestsadraw_bestsadmvsad_err_cost_(start_mv,mv_cost_params);// Search all possible scales up to the search param around the center point// pick the scale of the point that is best as the starting scale of// further steps around it.//初始搜索外层参数可选配置constuint8_t*center_addressget_buf_from_fullmv(ref,start_mv);//指向参考帧中对应于初始运动矢量 (start_mv) 的像素位置if(do_init_search){sbest_init_s;best_init_s-1;//遍历不同的搜索步长for(t0;ts;t){intbest_site-1;FULLPEL_MV center_mv{br,bc};//确定运动矢量中心if(check_bounds(ms_params-mv_limits,br,bc,1t)){// Call 4-point sad for multiples of 4 candidates.constintno_of_4_cand_loopsnum_candidates[t]2;//将候选点分成每组 4 个的组//在每个步长上候选点按照 4 个为一组计算 sad 并更新最佳 mvfor(i0;ino_of_4_cand_loops;i){calc_sad4_update_bestmv(ms_params,mv_cost_params,best_mv,center_mv,center_address,bestsad,raw_bestsad,t,best_site,i*4,/*cost_list*/NULL);//一次计算 4 个候选点的 SAD}// Rest of the candidates 剩余的候选点逐一计算 sad 并更新最佳 mvconstintremaining_candnum_candidates[t]%4;//不能整除 4 的剩余候选点calc_sad_update_bestmv(ms_params,mv_cost_params,best_mv,center_mv,center_address,bestsad,raw_bestsad,t,best_site,remaining_cand,no_of_4_cand_loops*4,NULL);}else{//边界以外的候选点逐个处理calc_sad_update_bestmv(ms_params,mv_cost_params,best_mv,center_mv,center_address,bestsad,raw_bestsad,t,best_site,num_candidates[t],0,NULL);}//选择最佳位置if(best_site-1){continue;}else{best_init_st;kbest_site;}}//更新最佳位置的运动矢量信息if(best_init_s!-1){brsearch_sites-site[best_init_s][k].mv.row;bcsearch_sites-site[best_init_s][k].mv.col;center_addresssearch_sites-site[best_init_s][k].offset;}}// If the center point is still the best, just skip this and move to// the refinement step.//如果中心点仍然是最佳的跳过这步骤移动到精细化步骤if(best_init_s!-1){constintlast_s(last_is_4cost_list!NULL);//布尔值表达式目的是决定精细化搜索的最低层次是否需要进一步处理intbest_site-1;sbest_init_s;//如果 best_init_s 是有效值算法将从该层开始向更细的层次逐步搜索//逐层优化从粗到细last_s 为 1 就循环 2 次为 0 就循环 1 次for(;slast_s;s--){// No need to search all points the 1st time if initial search was usedif(!do_init_search||s!best_init_s){FULLPEL_MV center_mv{br,bc};//边界检查检查候选店是否在允许的范围内if(check_bounds(ms_params-mv_limits,br,bc,1s)){// Call 4-point sad for multiples of 4 candidates.//对候选点进行批量搜索constintno_of_4_cand_loopsnum_candidates[s]2;for(i0;ino_of_4_cand_loops;i){calc_sad4_update_bestmv(ms_params,mv_cost_params,best_mv,center_mv,center_address,bestsad,raw_bestsad,s,best_site,i*4,/*cost_list*/NULL);}// Rest of the candidates 剩余候选点处理constintremaining_candnum_candidates[s]%4;calc_sad_update_bestmv(ms_params,mv_cost_params,best_mv,center_mv,center_address,bestsad,raw_bestsad,s,best_site,remaining_cand,no_of_4_cand_loops*4,NULL);}else{//候选点未通过边界检查直接处理所有候选点calc_sad_update_bestmv(ms_params,mv_cost_params,best_mv,center_mv,center_address,bestsad,raw_bestsad,s,best_site,num_candidates[s],0,NULL);}//更新最佳候选点找到最佳候选点 mv 和中心地址if(best_site-1){continue;}else{brsearch_sites-site[s][best_site].mv.row;bcsearch_sites-site[s][best_site].mv.col;center_addresssearch_sites-site[s][best_site].offset;kbest_site;}}//精细化处理do{//初始化检查点索引intnext_chkpts_indices[PATTERN_CANDIDATES_REF];best_site-1;next_chkpts_indices[0](k0)?num_candidates[s]-1:k-1;next_chkpts_indices[1]k;next_chkpts_indices[2](knum_candidates[s]-1)?0:k1;FULLPEL_MV center_mv{br,bc};//边界检查if(check_bounds(ms_params-mv_limits,br,bc,1s)){//对 3 个候选点进行 SAD 计算并更新最佳候选点calc_sad3_update_bestmv(ms_params,mv_cost_params,best_mv,center_mv,center_address,bestsad,raw_bestsad,s,best_site,next_chkpts_indices,NULL);}else{//边界外的候选点采用备用的方式更新最佳点calc_sad_update_bestmv_with_indices(ms_params,mv_cost_params,best_mv,center_mv,center_address,bestsad,raw_bestsad,s,best_site,PATTERN_CANDIDATES_REF,next_chkpts_indices,NULL);}//更新最佳候选点if(best_site!-1){knext_chkpts_indices[best_site];brsearch_sites-site[s][k].mv.row;bcsearch_sites-site[s][k].mv.col;center_addresssearch_sites-site[s][k].offset;}}while(best_site!-1);//直到没有更优的候选点退出循环}// Note: If we enter the if below, then cost_list must be non-NULL.//利用 cost_list 和相邻点优化搜索效率if(s0){cost_list[0]raw_bestsad;costlist_has_sad1;assert(num_candidates[s]4);if(!do_init_search||s!best_init_s){FULLPEL_MV center_mv{br,bc};//检查边界与候选点更新if(check_bounds(ms_params-mv_limits,br,bc,1s)){calc_sad4_update_bestmv(ms_params,mv_cost_params,best_mv,center_mv,center_address,bestsad,raw_bestsad,s,best_site,0,cost_list);}else{calc_sad_update_bestmv(ms_params,mv_cost_params,best_mv,center_mv,center_address,bestsad,raw_bestsad,s,best_site,/*num_candidates*/4,/*cand_start*/0,cost_list);}// 更新最佳运动矢量与中心地址if(best_site!-1){brsearch_sites-site[s][best_site].mv.row;bcsearch_sites-site[s][best_site].mv.col;center_addresssearch_sites-site[s][best_site].offset;kbest_site;}}//循环处理相邻候选点while(best_site!-1){intnext_chkpts_indices[PATTERN_CANDIDATES_REF];best_site-1;next_chkpts_indices[0](k0)?num_candidates[s]-1:k-1;next_chkpts_indices[1]k;next_chkpts_indices[2](knum_candidates[s]-1)?0:k1;cost_list[1]cost_list[2]cost_list[3]cost_list[4]INT_MAX;cost_list[((k2)%4)1]cost_list[0];cost_list[0]raw_bestsad;FULLPEL_MV center_mv{br,bc};//处理 3 个候选点的 sad 值并更新最佳候选点超过边界则采用备用方式更新if(check_bounds(ms_params-mv_limits,br,bc,1s)){assert(PATTERN_CANDIDATES_REF3);calc_sad3_update_bestmv(ms_params,mv_cost_params,best_mv,center_mv,center_address,bestsad,raw_bestsad,s,best_site,next_chkpts_indices,cost_list);}else{calc_sad_update_bestmv_with_indices(ms_params,mv_cost_params,best_mv,center_mv,center_address,bestsad,raw_bestsad,s,best_site,PATTERN_CANDIDATES_REF,next_chkpts_indices,cost_list);}//更新最佳 mv 信息if(best_site!-1){knext_chkpts_indices[best_site];brsearch_sites-site[s][k].mv.row;bcsearch_sites-site[s][k].mv.col;center_addresssearch_sites-site[s][k].offset;}}}}//更新最佳运动矢量best_mv-rowbr;best_mv-colbc;//同步性检查检查当前的 center_address 是否与 best_mv 计算得到的内存地址一致assert(center_addressget_buf_from_fullmv(ref,best_mv)center address is out of sync with best_mv!\n);// Returns the one-away integer pel cost/sad around the best as follows:// cost_list[0]: cost/sad at the best integer pel 最佳整像素点的 cost/sad// cost_list[1]: cost/sad at delta {0, -1} (left) from the best integer pel //左侧// cost_list[2]: cost/sad at delta { 1, 0} (bottom) from the best integer pel //底部// cost_list[3]: cost/sad at delta { 0, 1} (right) from the best integer pel //右边// cost_list[4]: cost/sad at delta {-1, 0} (top) from the best integer pel //顶部if(cost_list){if(USE_SAD_COSTLIST){calc_int_sad_list(*best_mv,ms_params,cost_list,costlist_has_sad);//计算 SAD 列表}else{calc_int_cost_list(*best_mv,ms_params,cost_list);//计算综合成本列表}}//获取最佳矢量的预测方差成本constintvar_costget_mvpred_var_cost(ms_params,best_mv,best_mv_stats);//返回最佳预测方差成本returnvar_cost;}