20M 文件用 Java 压缩从30秒到1秒的优化过程
public static void zipFileNoBuffer() {
File zipFile = new File(ZIP_FILE);
try (ZipOutputStream zipOut = new ZipOutputStream(new FileOutputStream(zipFile))) {
//开始时间
long beginTime = System.currentTimeMillis();
for (int i = 0; i < 10; i++) {
try (InputStream input = new FileInputStream(JPG_FILE)) {
zipOut.putNextEntry(new ZipEntry(FILE_NAME + i));
int temp = 0;
while ((temp = input.read()) != -1) {
zipOut.write(temp);}}}
printInfo(beginTime);}
catch (Exception e) {
e.printStackTrace();}}
fileSize:20M
consum time:29599
第一次优化过程-从30秒到2秒
/**
* Reads a byte of data from this input stream. This method blocks
* if no input is yet available.
*
* @return the next byte of data, or <code>-1</code> if the end of the
* file is reached.
* @exception IOException if an I/O error occurs.
*/
public native int read() throws IOException;
BufferedInputStream内部封装了一个byte数组用于存放数据,默认大小是8192
public static void zipFileBuffer() {
File zipFile = new File(ZIP_FILE);
try (ZipOutputStream zipOut = new ZipOutputStream(new FileOutputStream(zipFile));
BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(zipOut)) {
//开始时间
long beginTime = System.currentTimeMillis();
for (int i = 0; i < 10; i++) {
try (BufferedInputStream bufferedInputStream = new BufferedInputStream(new FileInputStream(JPG_FILE))) {
zipOut.putNextEntry(new ZipEntry(FILE_NAME + i));
int temp = 0;
while ((temp = bufferedInputStream.read()) != -1) {
bufferedOutputStream.write(temp);}}}
printInfo(beginTime);} catch (Exception e) {
e.printStackTrace();}}
------Buffer
fileSize:20M
consum time:1808
第二次优化过程-从2秒到1秒
使用Channel
public static void zipFileChannel() {
//开始时间
long beginTime = System.currentTimeMillis();
File zipFile = new File(ZIP_FILE);
try (ZipOutputStream zipOut = new ZipOutputStream(new FileOutputStream(zipFile));
WritableByteChannel writableByteChannel = Channels.newChannel(zipOut)) {
for (int i = 0; i < 10; i++) {
try (FileChannel fileChannel = new FileInputStream(JPG_FILE).getChannel()) {
zipOut.putNextEntry(new ZipEntry(i + SUFFIX_FILE));
fileChannel.transferTo(0, FILE_SIZE, writableByteChannel);}}
printInfo(beginTime);} catch (Exception e) {
e.printStackTrace();}}
This method is potentially much more efficient than a simple loop
* that reads from this channel and writes to the target channel. Many
* operating systems can transfer bytes directly from the filesystem cache
* to the target channel without actually copying them.
copy阶段就是从内核空间转到用户空间的一个过程
------Channel
fileSize:20M
consum time:1416
内核空间和用户空间
用户态、内核态:线程处于内核空间称之为内核态,线程处于用户空间属于用户态
直接缓冲区和非直接缓冲区
![](https://upload-images.jianshu.io/upload_images/13146186-6bbfb5446221cb2d.jpg?imageMogr2/auto-orient/strip|imageView2/2/w/812/format/webp)
image
综上所述,所以我们使用 transferTo方法就是直接开辟了一段直接缓冲区。所以性能相比而言提高了许多
使用内存映射文件
//Version 4 使用Map映射文件
public static void zipFileMap() {
//开始时间
long beginTime = System.currentTimeMillis();
File zipFile = new File(ZIP_FILE);
try (ZipOutputStream zipOut = new ZipOutputStream(new FileOutputStream(zipFile));
WritableByteChannel writableByteChannel = Channels.newChannel(zipOut)) {
for (int i = 0; i < 10; i++) {
zipOut.putNextEntry(new ZipEntry(i + SUFFIX_FILE));
//内存中的映射文件
MappedByteBuffer mappedByteBuffer = new RandomAccessFile(JPG_FILE_PATH, "r").getChannel().map(FileChannel.MapMode.READ_ONLY, 0, FILE_SIZE);
writableByteChannel.write(mappedByteBuffer);}
printInfo(beginTime);} catch (Exception e) {
e.printStackTrace();}}
---------Map
fileSize:20M
consum time:1305
使用Pipe
Whether or not a thread writing bytes to a pipe will block until another thread reads those bytes
//Version 5 使用Pip
public static void zipFilePip() {
long beginTime = System.currentTimeMillis();
try(WritableByteChannel out = Channels.newChannel(new FileOutputStream(ZIP_FILE))) {
Pipe pipe = Pipe.open();
//异步任务
CompletableFuture.runAsync(()->runTask(pipe));
//获取读通道
ReadableByteChannel readableByteChannel = pipe.source();
ByteBuffer buffer = ByteBuffer.allocate(((int) FILE_SIZE)*10);
while (readableByteChannel.read(buffer)>= 0) {
buffer.flip();
out.write(buffer);
buffer.clear();}}
catch (Exception e){
e.printStackTrace();}
printInfo(beginTime);
}
//异步任务
public static void runTask(Pipe pipe) {
try(ZipOutputStream zos = new ZipOutputStream(Channels.newOutputStream(pipe.sink()));
WritableByteChannel out = Channels.newChannel(zos)) {
System.out.println("Begin");
for (int i = 0; i < 10; i++) {
zos.putNextEntry(new ZipEntry(i+SUFFIX_FILE));
FileChannel jpgChannel = new FileInputStream(new File(JPG_FILE_PATH)).getChannel();
jpgChannel.transferTo(0, FILE_SIZE, out);
jpgChannel.close();}}
catch (Exception e){
e.printStackTrace();}}
总结
参考文章
来源:https://www.jianshu.com/p/25b328753017
-END-