20M 文件用 Java 压缩从30秒到1秒的优化过程
public static void zipFileNoBuffer() {File zipFile = new File(ZIP_FILE);try (ZipOutputStream zipOut = new ZipOutputStream(new FileOutputStream(zipFile))) {//开始时间long beginTime = System.currentTimeMillis();
for (int i = 0; i < 10; i++) {try (InputStream input = new FileInputStream(JPG_FILE)) { zipOut.putNextEntry(new ZipEntry(FILE_NAME + i));int temp = 0;while ((temp = input.read()) != -1) { zipOut.write(temp);}}} printInfo(beginTime);} catch (Exception e) { e.printStackTrace();}}
fileSize:20Mconsum time:29599
第一次优化过程-从30秒到2秒
/*** Reads a byte of data from this input stream. This method blocks * if no input is yet available. * * @return the next byte of data, or <code>-1</code> if the end of the * file is reached. * @exception IOException if an I/O error occurs. */public native int read() throws IOException;
BufferedInputStream内部封装了一个byte数组用于存放数据,默认大小是8192
public static void zipFileBuffer() {File zipFile = new File(ZIP_FILE);try (ZipOutputStream zipOut = new ZipOutputStream(new FileOutputStream(zipFile));BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(zipOut)) {//开始时间long beginTime = System.currentTimeMillis();for (int i = 0; i < 10; i++) {try (BufferedInputStream bufferedInputStream = new BufferedInputStream(new FileInputStream(JPG_FILE))) { zipOut.putNextEntry(new ZipEntry(FILE_NAME + i));int temp = 0;while ((temp = bufferedInputStream.read()) != -1) { bufferedOutputStream.write(temp);}}} printInfo(beginTime);} catch (Exception e) { e.printStackTrace();}}
------BufferfileSize:20Mconsum time:1808
第二次优化过程-从2秒到1秒
使用Channel
public static void zipFileChannel() {//开始时间long beginTime = System.currentTimeMillis();File zipFile = new File(ZIP_FILE);try (ZipOutputStream zipOut = new ZipOutputStream(new FileOutputStream(zipFile));WritableByteChannel writableByteChannel = Channels.newChannel(zipOut)) {for (int i = 0; i < 10; i++) {try (FileChannel fileChannel = new FileInputStream(JPG_FILE).getChannel()) { zipOut.putNextEntry(new ZipEntry(i + SUFFIX_FILE)); fileChannel.transferTo(0, FILE_SIZE, writableByteChannel);}} printInfo(beginTime);} catch (Exception e) { e.printStackTrace();}}
This method is potentially much more efficient than a simple loop* that reads from this channel and writes to the target channel. Many* operating systems can transfer bytes directly from the filesystem cache* to the target channel without actually copying them.
copy阶段就是从内核空间转到用户空间的一个过程
------ChannelfileSize:20Mconsum time:1416
内核空间和用户空间
用户态、内核态:线程处于内核空间称之为内核态,线程处于用户空间属于用户态
直接缓冲区和非直接缓冲区
image
综上所述,所以我们使用 transferTo方法就是直接开辟了一段直接缓冲区。所以性能相比而言提高了许多
使用内存映射文件
//Version 4 使用Map映射文件public static void zipFileMap() {//开始时间long beginTime = System.currentTimeMillis();File zipFile = new File(ZIP_FILE);try (ZipOutputStream zipOut = new ZipOutputStream(new FileOutputStream(zipFile));WritableByteChannel writableByteChannel = Channels.newChannel(zipOut)) {for (int i = 0; i < 10; i++) { zipOut.putNextEntry(new ZipEntry(i + SUFFIX_FILE));//内存中的映射文件MappedByteBuffer mappedByteBuffer = new RandomAccessFile(JPG_FILE_PATH, "r").getChannel().map(FileChannel.MapMode.READ_ONLY, 0, FILE_SIZE); writableByteChannel.write(mappedByteBuffer);} printInfo(beginTime);} catch (Exception e) { e.printStackTrace();}}
---------MapfileSize:20Mconsum time:1305
使用Pipe
Whether or not a thread writing bytes to a pipe will block until another thread reads those bytes
//Version 5 使用Pippublic static void zipFilePip() {long beginTime = System.currentTimeMillis();try(WritableByteChannel out = Channels.newChannel(new FileOutputStream(ZIP_FILE))) {Pipe pipe = Pipe.open();//异步任务CompletableFuture.runAsync(()->runTask(pipe));
//获取读通道ReadableByteChannel readableByteChannel = pipe.source();ByteBuffer buffer = ByteBuffer.allocate(((int) FILE_SIZE)*10);while (readableByteChannel.read(buffer)>= 0) { buffer.flip();out.write(buffer); buffer.clear();}}catch (Exception e){ e.printStackTrace();} printInfo(beginTime);}//异步任务public static void runTask(Pipe pipe) {try(ZipOutputStream zos = new ZipOutputStream(Channels.newOutputStream(pipe.sink()));WritableByteChannel out = Channels.newChannel(zos)) {System.out.println("Begin");for (int i = 0; i < 10; i++) { zos.putNextEntry(new ZipEntry(i+SUFFIX_FILE));FileChannel jpgChannel = new FileInputStream(new File(JPG_FILE_PATH)).getChannel(); jpgChannel.transferTo(0, FILE_SIZE, out); jpgChannel.close();}} catch (Exception e){ e.printStackTrace();}}
总结
参考文章
来源:https://www.jianshu.com/p/25b328753017
-END-
