scipy.sparse矩阵(csr_matrix())但我需要将它保存到一个文件，而不是.npz格式，而是作为一个常规的.txt或.csv文件.我的问题是，我没有足够的内存将稀疏矩阵转换成常规的np.array()，然后保存到文件中.是否有一种方法可以将数据作为稀疏矩阵存储在内存中，但直接将其保存为正则矩阵，格式如下:
0 0 0
0 1 0
1 0 1
到磁盘上？或者，有没有一种方法可以在Python内部不加载到内存中的情况下"解压缩"一个.npz文件？(例如Bash中的gunzip或unzip).
对新问题的回答:
import numpy as np
from scipy import sparse, io
A = sparse.eye(5, format='csr') * np.pi
np.set_printoptions(precision=16, linewidth=1000)
with open('matrix.txt', 'a') as f:
for row in A:
f.write(str(row.toarray()[0]))
f.write('\n')
# [3.141592653589793 0. 0. 0. 0. ]
# [0. 3.141592653589793 0. 0. 0. ]
# [0. 0. 3.141592653589793 0. 0. ]
# [0. 0. 0. 3.141592653589793 0. ]
# [0. 0. 0. 0. 3.141592653589793]
并加上开始/结束括号:
import numpy as np
from scipy import sparse, io
A = sparse.eye(5, format='csr') * np.pi
np.set_printoptions(precision=16, linewidth=1000)
with open('matrix.txt', 'a') as f:
for i, row in enumerate(A):
f.write('[' if (i == 0) else ' ')
f.write(str(row.toarray()[0]))
f.write(']' if (i == A.shape[0] - 1) else '\n')
# [[3.141592653589793 0. 0. 0. 0. ]
# [0. 3.141592653589793 0. 0. 0. ]
# [0. 0. 3.141592653589793 0. 0. ]
# [0. 0. 0. 3.141592653589793 0. ]
# [0. 0. 0. 0. 3.141592653589793]]
您可能需要根据您的数据调整set_printoptions个.
回答原始问题，这不要求矩阵写得密集.
import numpy as np
from scipy import sparse, io
A = sparse.eye(3, format='csr') * np.pi
# Default title 0
# 3 1 1 1
# RUA 3 3 3 0
# (40I2) (40I2) (3E25.16)
# 1 2 3 4
# 1 2 3
# 3.1415926535897931E+00 3.1415926535897931E+00 3.1415926535897931E+00
io.hb_write('matrix.txt', A) # saves as matrix.txt
A2 = io.hb_read('matrix.txt')
assert not (A2 != A).nnz # efficient check for equality
io.mmwrite('matrix', A) # saves as matrix.mtx
# %%MatrixMarket matrix coordinate real symmetric
# %
# 3 3 3
# 1 1 3.141592653589793e+00
# 2 2 3.141592653589793e+00
# 3 3 3.141592653589793e+00
A2 = io.mmread('matrix')
assert not (A2 != A).nnz
如果你想要一个更简单的format，尽管它涉及更多的代码:
import numpy as np
from scipy import sparse
A = sparse.eye(10, format='csr')*np.pi
np.savetxt('data.txt', A.data)
np.savetxt('indices.txt', A.indices, fmt='%i')
np.savetxt('indptr.txt', A.indptr, fmt='%i')
加载:
data = np.loadtxt('data.txt')
indices = np.loadtxt('indices.txt', dtype=np.int32)
indptr = np.loadtxt('indptr.txt', dtype=np.int32)
A2 = sparse.csr_matrix((data, indices, indptr))
assert not (A2 != A).nnz
但重要的是，你需要保存的只是csr_matrix的data、indices和indptr属性.