This file lists some rules that must be followed by kernel developers (i.e., programmers that use auto-mode compiler and pto-isa library to write their own kernels).【免费下载链接】pto-isaParallel Tile Operation (PTO) is a virtual instruction set architecture designed by Ascend CANN, focusing on tile-level operations. This repository offers high-performance, cross-platform tile operations across Ascend platforms.项目地址: https://gitcode.com/cann/pto-isaNot following these rules can lead to any of the following consequences:Fail to compile (either from source code level or crash in the compiler)Functionally incorrect (e.g., precision issues)Bad performance1 - Control Flow RulesComplex control flow (especially inside loops) often makes it difficult to optimize the precise cross-pipe parallelization and double-buffering. Since PTO AUTO compiler is required to maintain program correctness, it may generate the sync operations more conservatively leading to performance degradation.1.1 - Guards for First and Last IteraionsAny condition that guards the first and last iteraion of a loop should be expressed in a form that can be statically evaluated. That makes it possible for the PTO AUTO compiler to automatically peel peel the first and last iteration of the loop which results in simplifying the auto synchronization substantially. Here is an example:for (int tile_id 0; tile_id total_tiles; tile_id) { if (tile_id 0) { TLOAD(srcTile, globalSrc); } ... if (tile_id total_tiles-1) { TSTORE(globalDst, dstTile); } }1.2 - Loop Invariant Control Flow in Loop Nestsif-statements Nested in a Loop that do not depend on inner loops induction variable should be left inside the inner loop.For example, consider the if-statement inside the inner loop:for (int tile_id 0; tile_id total_tiles; tile_id) { int next_tile tile_id total_tiles-1 ? tile_id 1 : -1; ... for (int subtile_id 0; subtile_id total_subtiles; subtile_id) { if (next_tile ! -1) { ... // computation here } } }This should instead be rewritten asfor (int tile_id 0; tile_id total_tiles; tile_id) { int next_tile tile_id total_tiles-1 ? tile_id 1 : -1; ... if (next_tile ! -1) { for (int subtile_id 0; subtile_id total_subtiles; subtile_id) { ... // computation here } } }1.3 - Complex Logical Expressions in If StatementsIt is highly recommended that complex logical expressions that guard PTO instructions are evaluated prior to being used in the if-statement.As an example, consider the following if-statement:if ((srcTile.GetValidRow() 16 || srcTile.GetValidCol() 16) srcTile.GetKAligned()) { TLOAD(srcTile, globalSrc1); } else { TLOAD(srcTile, globalSrc0); }It is recommended to be rewritten to the following form:bool cond (srcTile.GetValidRow() 16 || srcTile.GetValidCol() 16) srcTile.GetKAligned(); if (cond) { TLOAD(srcTile, globalSrc1); } else { TLOAD(srcTile, globalSrc0); }1.4 - Its strongly recommended NOT to use double/multi buffering at the momentFor now, the double/multi buffering in auto mode isnt fully supported, because once the kernels becomes complicated, using double buffering always involves complex control flows, imposing huges challenges for compilers to do auto-sync. Auto mode compiler team is trying to design a dedicated abstraction/interface (with some constraints) to kernel developers, to enable double buffering, while the compiler can correctly analyze the code and do proper auto sync.2 - Memory Allocation Rules2.1 UseTRESHAPEto tell compiler that 2 tiles have the same base addressIf you want to express that tile B has the same base address as tile A (alias), you can useTRESHAPEwhich works for both manual and auto mode, instead of usingTASSIGNdirectly. For instance:TileSrcTypeA tileA; TileSrcTypeB tileB; // Invalid in auto mode TASSIGN(tileA, 0x0); TASSIGN(tileB, 0x0); // Correct in auto mode TRESHAPE(tileB, tileA);2.2 UseTSUBVIEWto tell compiler that tile B is a sub-view of tile A ***Same purpose asTRESHAPE, but this is used to express that tile Bs address is based on tile As address, plus some offset. This interface is needed for auto mode because it tells the compiler how 2 tiles alias with each other. You can checkdocs/isa/TSUBVIEW.mdfor details.Example:uint16_t rowOffset, colOffset; // can be runtime variable // addr(tileB) addr(tileA) offsets TileData tileA(...); TileData tileB(...); // Invalid in auto mode TASSIGN(tileA, 0x0); TASSIGN(tileB, 0x0 rowOffset * TileData::Col colOffset * 1 sizeof(T)); // Correct for auto mode TSUBVIEW(tileB, tileA, rowOffset, colOffset);2.3 - Keep in mind that tiles memory shouldnt change at runtime in auto modePTO AUTO compiler automatically assigns constant memory addresses to each declared tile variable. That means, unlike in PTO manual mode, tile memory addresses cannot change in the middle of a kernel. For example, consider the following PTO Manual code:TileData tile; for (int i 0; i N; i) { TASSIGN(tile, 0x100 * i); foo(tile); }Users in manual mode can use trick like this to change a tile address at any point during runtime. Compiler cant handle such dynamic behavior to properly allocate memory;In auto mode, each tile will be allocated memory once and for all.Programmers should rewrite their code to bypass such limitation.Heres a crucial mindset for auto mode:think of a tile as a C reference, meaning that its memory address is already determined and cannot change once declared.2.4 - Correctly understand the semantics ofTRESHAPEandTSUBVIEWIn manual mode, they are both actual PTO instructions that (re)assign an address to a tile. However, their semantic is different in auto mode:they simply serve as a hint to the compiler about how 2 tiles alias with each other.TRESHAPE:The semantic of theTRESHAPEinstruction is slightly different in the AUTO mode compared to the Manual Mode. In Manual Mode,TRESHAPEit assigns the address of thesourcetile to thedestinationtile at the point of execution of theTRESHAPEinstruction. However, in the AUTO mode,TRESHAPEacts as a mechanism to bind the source and destination tile to the same address. This binding is valid across the entire scope in which the source and destination tiles are defined.TSUBVIEW:TheTSUBVIEWinstruction allows users obtain a subtile from a larger tile. In the AUTO mode, the compiler calculates the relative offset of the subtile and add it to the automatically allocated address of the base tile.Note that since in the AUTO mode is that the address of a tile cannot change throughout its scope, a tile cannot be used as the destination for multipleTRESHAPEorTSUBVIEWinstructions. For example, the following example is invalid in the AUTO mode and the actual behavior is undefined.TRESHAPE(tile0, tile1); foo(tile0); ... TSUBVIEW(tile0, tile2, 0, 0); bar(tile0);As a good practice, it is highly recommended that theTSUBVIEWandTRESHAPEinstructions are placed right the declaration of the destination tiles.3 - General rules3.1 - Dont callTLOADon a destination tileIf a tile is supposed to be used as output and has no copied-in data from GM, you should never callTLOADon it. First: its unnecessary; Second: it may cause data races in auto mode.Take this example:TLOAD(dstTile, dstGlobal); // redundant TLOAD(srcTile, srcGlobal); TEXP(dstTile, srcTile); TSTORE(dstGlobal, dstTile);In manual mode, programmers can assignsrcTileanddstTilewith different addresses manually, and this works perfectly fine.However, in auto mode, this may be problematic: here the liveness range ofdstTileandsrcTilewont overlap, thus the compiler may reuse the memory by assigning them to the same address. If theyre both called withTLOAD, then these 2TLOADwill occur at the same time and as a consequence, they overwrite each other.DONT CALL REDUNDANTTLOAD!3.2 - Dont call CCE intrinsics directlyKernel developers using PTO should call PTO instructions only; they shouldnt call CCE intrinsics directly. There are 2 reasons:CCE intrinsics take raw pointer as arguments, which wont compile for auto Mode (since tile is represented as a vector type instead of pointer type in auto mode; seepto-isa/include/pto/common/memory.hppfor details).Automatic memory allocation and synchronization is built upon the analysis of PTO instructions only; they cant recognize anything else.For this reason, you shouldnt call.data()member function of a tile directly in your kernel; technically speaking, this is NOT an interface to be called by the kernel developers, it should only be used for library developers instead.If you have no choice (e.g., theres no equivalent of a PTO instruction) but use CCE intrinsics directly, you should summit a request to pto-isa to add new PTO instruction.3.3 - Prefer usingPtoSetWaitFlagorTSYNCinstead ofset_flagandwait_flagInternal implementation ofPtoSetWaitFlagandTSYNChave guards against manual and auto mode; when compiling in auto mode, this interface is a no-op, so that it doesnt conflict with auto-sync from compiler. This can save you some typing and make the code look cleaner.If you callset_flagandwait_flagdirectly in your kernel, you always need to manually guard it using the macro__PTO_AUTO__, which is cumbersome.【免费下载链接】pto-isaParallel Tile Operation (PTO) is a virtual instruction set architecture designed by Ascend CANN, focusing on tile-level operations. This repository offers high-performance, cross-platform tile operations across Ascend platforms.项目地址: https://gitcode.com/cann/pto-isa创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考
CANN/pto-isa内核开发规则
This file lists some rules that must be followed by kernel developers (i.e., programmers that use auto-mode compiler and pto-isa library to write their own kernels).【免费下载链接】pto-isaParallel Tile Operation (PTO) is a virtual instruction set architecture designed by Ascend CANN, focusing on tile-level operations. This repository offers high-performance, cross-platform tile operations across Ascend platforms.项目地址: https://gitcode.com/cann/pto-isaNot following these rules can lead to any of the following consequences:Fail to compile (either from source code level or crash in the compiler)Functionally incorrect (e.g., precision issues)Bad performance1 - Control Flow RulesComplex control flow (especially inside loops) often makes it difficult to optimize the precise cross-pipe parallelization and double-buffering. Since PTO AUTO compiler is required to maintain program correctness, it may generate the sync operations more conservatively leading to performance degradation.1.1 - Guards for First and Last IteraionsAny condition that guards the first and last iteraion of a loop should be expressed in a form that can be statically evaluated. That makes it possible for the PTO AUTO compiler to automatically peel peel the first and last iteration of the loop which results in simplifying the auto synchronization substantially. Here is an example:for (int tile_id 0; tile_id total_tiles; tile_id) { if (tile_id 0) { TLOAD(srcTile, globalSrc); } ... if (tile_id total_tiles-1) { TSTORE(globalDst, dstTile); } }1.2 - Loop Invariant Control Flow in Loop Nestsif-statements Nested in a Loop that do not depend on inner loops induction variable should be left inside the inner loop.For example, consider the if-statement inside the inner loop:for (int tile_id 0; tile_id total_tiles; tile_id) { int next_tile tile_id total_tiles-1 ? tile_id 1 : -1; ... for (int subtile_id 0; subtile_id total_subtiles; subtile_id) { if (next_tile ! -1) { ... // computation here } } }This should instead be rewritten asfor (int tile_id 0; tile_id total_tiles; tile_id) { int next_tile tile_id total_tiles-1 ? tile_id 1 : -1; ... if (next_tile ! -1) { for (int subtile_id 0; subtile_id total_subtiles; subtile_id) { ... // computation here } } }1.3 - Complex Logical Expressions in If StatementsIt is highly recommended that complex logical expressions that guard PTO instructions are evaluated prior to being used in the if-statement.As an example, consider the following if-statement:if ((srcTile.GetValidRow() 16 || srcTile.GetValidCol() 16) srcTile.GetKAligned()) { TLOAD(srcTile, globalSrc1); } else { TLOAD(srcTile, globalSrc0); }It is recommended to be rewritten to the following form:bool cond (srcTile.GetValidRow() 16 || srcTile.GetValidCol() 16) srcTile.GetKAligned(); if (cond) { TLOAD(srcTile, globalSrc1); } else { TLOAD(srcTile, globalSrc0); }1.4 - Its strongly recommended NOT to use double/multi buffering at the momentFor now, the double/multi buffering in auto mode isnt fully supported, because once the kernels becomes complicated, using double buffering always involves complex control flows, imposing huges challenges for compilers to do auto-sync. Auto mode compiler team is trying to design a dedicated abstraction/interface (with some constraints) to kernel developers, to enable double buffering, while the compiler can correctly analyze the code and do proper auto sync.2 - Memory Allocation Rules2.1 UseTRESHAPEto tell compiler that 2 tiles have the same base addressIf you want to express that tile B has the same base address as tile A (alias), you can useTRESHAPEwhich works for both manual and auto mode, instead of usingTASSIGNdirectly. For instance:TileSrcTypeA tileA; TileSrcTypeB tileB; // Invalid in auto mode TASSIGN(tileA, 0x0); TASSIGN(tileB, 0x0); // Correct in auto mode TRESHAPE(tileB, tileA);2.2 UseTSUBVIEWto tell compiler that tile B is a sub-view of tile A ***Same purpose asTRESHAPE, but this is used to express that tile Bs address is based on tile As address, plus some offset. This interface is needed for auto mode because it tells the compiler how 2 tiles alias with each other. You can checkdocs/isa/TSUBVIEW.mdfor details.Example:uint16_t rowOffset, colOffset; // can be runtime variable // addr(tileB) addr(tileA) offsets TileData tileA(...); TileData tileB(...); // Invalid in auto mode TASSIGN(tileA, 0x0); TASSIGN(tileB, 0x0 rowOffset * TileData::Col colOffset * 1 sizeof(T)); // Correct for auto mode TSUBVIEW(tileB, tileA, rowOffset, colOffset);2.3 - Keep in mind that tiles memory shouldnt change at runtime in auto modePTO AUTO compiler automatically assigns constant memory addresses to each declared tile variable. That means, unlike in PTO manual mode, tile memory addresses cannot change in the middle of a kernel. For example, consider the following PTO Manual code:TileData tile; for (int i 0; i N; i) { TASSIGN(tile, 0x100 * i); foo(tile); }Users in manual mode can use trick like this to change a tile address at any point during runtime. Compiler cant handle such dynamic behavior to properly allocate memory;In auto mode, each tile will be allocated memory once and for all.Programmers should rewrite their code to bypass such limitation.Heres a crucial mindset for auto mode:think of a tile as a C reference, meaning that its memory address is already determined and cannot change once declared.2.4 - Correctly understand the semantics ofTRESHAPEandTSUBVIEWIn manual mode, they are both actual PTO instructions that (re)assign an address to a tile. However, their semantic is different in auto mode:they simply serve as a hint to the compiler about how 2 tiles alias with each other.TRESHAPE:The semantic of theTRESHAPEinstruction is slightly different in the AUTO mode compared to the Manual Mode. In Manual Mode,TRESHAPEit assigns the address of thesourcetile to thedestinationtile at the point of execution of theTRESHAPEinstruction. However, in the AUTO mode,TRESHAPEacts as a mechanism to bind the source and destination tile to the same address. This binding is valid across the entire scope in which the source and destination tiles are defined.TSUBVIEW:TheTSUBVIEWinstruction allows users obtain a subtile from a larger tile. In the AUTO mode, the compiler calculates the relative offset of the subtile and add it to the automatically allocated address of the base tile.Note that since in the AUTO mode is that the address of a tile cannot change throughout its scope, a tile cannot be used as the destination for multipleTRESHAPEorTSUBVIEWinstructions. For example, the following example is invalid in the AUTO mode and the actual behavior is undefined.TRESHAPE(tile0, tile1); foo(tile0); ... TSUBVIEW(tile0, tile2, 0, 0); bar(tile0);As a good practice, it is highly recommended that theTSUBVIEWandTRESHAPEinstructions are placed right the declaration of the destination tiles.3 - General rules3.1 - Dont callTLOADon a destination tileIf a tile is supposed to be used as output and has no copied-in data from GM, you should never callTLOADon it. First: its unnecessary; Second: it may cause data races in auto mode.Take this example:TLOAD(dstTile, dstGlobal); // redundant TLOAD(srcTile, srcGlobal); TEXP(dstTile, srcTile); TSTORE(dstGlobal, dstTile);In manual mode, programmers can assignsrcTileanddstTilewith different addresses manually, and this works perfectly fine.However, in auto mode, this may be problematic: here the liveness range ofdstTileandsrcTilewont overlap, thus the compiler may reuse the memory by assigning them to the same address. If theyre both called withTLOAD, then these 2TLOADwill occur at the same time and as a consequence, they overwrite each other.DONT CALL REDUNDANTTLOAD!3.2 - Dont call CCE intrinsics directlyKernel developers using PTO should call PTO instructions only; they shouldnt call CCE intrinsics directly. There are 2 reasons:CCE intrinsics take raw pointer as arguments, which wont compile for auto Mode (since tile is represented as a vector type instead of pointer type in auto mode; seepto-isa/include/pto/common/memory.hppfor details).Automatic memory allocation and synchronization is built upon the analysis of PTO instructions only; they cant recognize anything else.For this reason, you shouldnt call.data()member function of a tile directly in your kernel; technically speaking, this is NOT an interface to be called by the kernel developers, it should only be used for library developers instead.If you have no choice (e.g., theres no equivalent of a PTO instruction) but use CCE intrinsics directly, you should summit a request to pto-isa to add new PTO instruction.3.3 - Prefer usingPtoSetWaitFlagorTSYNCinstead ofset_flagandwait_flagInternal implementation ofPtoSetWaitFlagandTSYNChave guards against manual and auto mode; when compiling in auto mode, this interface is a no-op, so that it doesnt conflict with auto-sync from compiler. This can save you some typing and make the code look cleaner.If you callset_flagandwait_flagdirectly in your kernel, you always need to manually guard it using the macro__PTO_AUTO__, which is cumbersome.【免费下载链接】pto-isaParallel Tile Operation (PTO) is a virtual instruction set architecture designed by Ascend CANN, focusing on tile-level operations. This repository offers high-performance, cross-platform tile operations across Ascend platforms.项目地址: https://gitcode.com/cann/pto-isa创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考
