Tags

follow dev59_3, this summarizes two bugs previously encountered on Ascend, referring to PyQCU/test/bug/bug26.md. :D

follow dev59_2, 1. some parameters were obtained from the re-testing on Ascend, referring to PyQCU/test/dev59_3; 2. the files in the `test` directory have been organized. :)

follow bug25, referring to PyQCU/test/test.ascend.*.npu*-dev59.*, the modified mg algorithm for npu just pass test, however, because this version had to address NPU precision and array dimension issues, as well as complex number incompatibility, it suffered significant performance losses. Hopefully, future versions will improve upon this. -_-

follow dev59_1, there is a bug in /root/PyQCU/pyqcu/ascend/inverse.py:npu_local_orthogonalize, the right part of this function is "    v = null_vecs.reshape(-1, Z, z, Y, y, X, x).clone()
    v = v.permute(0, 1, 3, 5, 2, 4, 6).contiguous()  # [EeTt,Z,Y,X,z,y,x]
    v = v.reshape(E, e, T, t, Z*Y*X, z*y*x).clone()
    v = v.permute(2, 4, 0, 1, 3, 5).contiguous()  # [T,ZYX,E,e,t,zyx]", referring to PyQCU/test/bug25.log. ^_^

follow dev59, 1. the modified mg algorithm adapted to address NPU precision and array dimensionality limitations has been implemented and tested in PyTorch, but it incurs a significant performance penalty; 2. the DCU compatibility issue has been identified as a problem on the DCU side, and a solution is being coordinated. R.I.P......

follow dev58, 1. the problem of convergence difficulty in single-precision processing of the MG algorithm was solved by using a simple method of layer truncation, referring to PyQCU/test/pyqcu/ascend/inverse.py:mg:adaptive; 2. independent settings for the precision of each layer and the device were implemented in the multi-threaded MG algorithm, referring to PyQCU/test/test.ascend.mg-dev59-np8.log; 3. the comparison with the wilson inverse results from quda was completed, referring to PyQCU/test/test.ascend.bistabcg-dev59.ipynb; however, the environment configuration for DCU and Ascend is not yet fully complete......

follow dev57, referring to PyQCU/test/*-dev58.*, the new multi-threaded PyTorch version of mg runs almost as fast as  multi-threaded PyTorch version of bistabcg, we need bigger lattice size and more threads to test, and it will be better and more obvious with gpus. doge :)

follow dev56, referring to PyQCU/test/*-dev57.*, multi-threaded PyTorch version of mg runs smooth now, but slower than multi-threaded PyTorch version of bistabcg, optimization is needed......

follow bug24, this bug has been clean!, multi-threaded PyTorch version of bistabcg runs perfect now! :)

follow dev55, I plan to implement a preliminary multi-threaded PyTorch version of mg in dev56 (with the finest level of matvec being parallel, and the rest being on the root thread, using mpi4py), however, I encountered a bug in the early testing of the multi-threaded PyTorch version of bistabcg(referring to PyQCU/test/bug24.log), hope to be able to successfully eliminate the bug later. orz

follow dev54_1, referring to PyQCU/test/test.ascend.mg-dev53.ipynb(changed, wilson version) and PyQCU/test/test.ascend.mg-dev55.ipynb(new, clover version), optimize PyQCU/pyqcu/ascend/inverse.py:mg:cycle by changing "x = torch.randn_like(b)" to "x = torch.zeros_like(b)" and "_tol = torch.norm(b).item()*0.01 if level != self.num_levels - 1 else torch.norm(b).item()*0.001" to "_tol = r_norm*0.25 if level != self.num_levels - 1 else r_norm*0.1"(at the same time, remove some prints), as a result, there is finally a real speedup in the time sense!!! :):):)

follow dev54, 1. do some format job to PyQCU/test/pyqcu/cuda, just like "def solver(b: cp_ndarray, matvec: Callable[[cp_ndarray], cp_ndarray], tol: float = 1e-6, max_iter: int = 1000, x0: cp_ndarray = None) -> cp_ndarray:
", just for easy to call functions in pyqcu.cuda; 2. add PyQCU/test/test-npX.x99-test5.sh & *.x99-test5.log, a recheck for "1". :)

follow test9_1, check and debug PyQCU/test/test.laplacian.py pass, "from opt_einsum import contract" can not be used in dcu("cupy_backends.cuda.libs.cublas.CUBLASError: HIPBLAS_STATUS_NOT_SUPPORTED")?

follow test9, referring to PyQCU/test/test.*-test*.log.json, nvprof test result saved.

follow test8, referring to PyQCU/test/test.*-test9.log, 1024 DCU test pass. :)

follow test5_1, referring to PyQCU/test/test.*-test8.log, 1024 DCU test pass. :)

follow test7, add PyQCU/test/test.*-test5.log, change PyQCU/test/test.*-test5.py,just a supplementary again.

follow test5, like test6, referring to PyQCU/test/test.clover.bistabcg-np*-test7.log, clover version of bistabcg passes in multiple node multithreading(16 & 32 &64)!!!!!! :):):)

follow test6, same as test6, just a supplementary......

follow dev53, 1. "gpus_per_node = cp.cuda.runtime.getDeviceCount(), local_rank = rank % gpus_per_node!!!" and "checkCudaErrors(cudaSetDevice(getLocalRank())); // !!!!!!"; 2. PyQCU/test/pyqcu/cuda/io.py:get_or_give & try_give add; 3. referring to PyQCU/test/test.clover.bistabcg-np4-test6.log & PyQCU/test/test.clover.bistabcg-np4-test6.log, clover version of bistabcg passes in single node multithreading(4 & 8); 4. 32,64,64,64,double,clover,bistabcg -> 64\*8\*0.15=76.8GB. ^_^