您当前的位置: > 详细浏览

SKA低频成像管线并行优化

Optimization of parallel processing of Square Kilometre Array low frequency imaging pipeline

摘要:平方公里阵列(Square Kilometre Array, SKA)射电望远镜的数据处理是通过管线方式进行的,管 线的执行效率是SKA区域中心考虑的重要因素.连续谱成像观测是SKA的主要观测模式之一,也是许 多科学工作的基础.本文以SKA低频先导设备(Murchison Widefield Array, MWA)的成像管线为例,在 中国SKA区域中心原型机(China SKA Regional Centre prototype, CSRC-P)上进行并行处理管线优化.以往的优化方案都集中在少数性能热点,缺乏对整体管线的系统优化,导致整体加速比相对较低.针对这一 问题,本文提出了一种全局优化方案,针对管线使用多种编程语言、图像数据可独立处理的特点,综合使用C+ +多线程、Python多进程、Shell多任务并行等优化方法,并验证了优化结果的准确性.实验表明,优 化后的代码在CSRC-P的x86节点和ARM节点上分别获得了2.7和2.4倍加速. ARM计算节点展现出 对SKA应用良好的适应性.本文的优化策略和方法也适用于其他SKA科学应用,对SKA先导望远镜的 科学运行和未来的运行有帮助.

英文摘要:Data processing of the Square Kilometre Array (SKA) is carried out in pipeline mode, and the execution efficiency of pipeline is an important factor to be considered in SKA data processing. Continuum imaging is one of the main observation modes of SKA and is a prerequisite for many other scientific works. In this paper, we take the imaging pipeline of SKA low-frequency precursor Murchison Widefield Array (MWA), as an example and optimize the parallel processing pipeline on the China SKA Regional Centre prototype (CSRC-P). Previous optimization schemes have focused on a few performance hotspots and lacked systematic optimization of the overall pipeline, resulting in a relatively poor overall speedup ratio. In this paper, we propose a global optimization scheme that combines C++ multi-threading, Python multi-processing, and Shell multi-tasking parallelism for pipelines using multiple programming languages and image datasets that can be processed independently, and verify the accuracy of the optimization results. Experiments show taht the optimized pipeline achieves an overall speedup of 2.7 and 2.4 times on the x86 and ARM nodes of CSRC-P, respectively, and the ARM compute nodes shows good adaptability to SKA applications. The optimization strategies and methods in this paper are also applicable to other SKA applications and will be useful for the scientific operation and future operation of the SKA precursor telescope.

版本历史

[V1] 2022-06-28 19:16:56 chinaXiv:202206.00187V1 下载全文
点击下载全文
同行评议状态
待评议
许可声明
metrics指标
  • 点击量1838
  • 下载量97
评论
分享
邀请专家评阅