** Flume Sink로 hdfs 연동 중 아래와 같은 오류 발생
2020-04-14 00:08:04,350 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:459)] process failed
java.lang.NoClassDefFoundError: com/ctc/wstx/io/InputBootstrapper
at org.apache.flume.sink.hdfs.BucketWriter.open(BucketWriter.java:219)
at org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:572)
at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:412)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.ClassNotFoundException: com.ctc.wstx.io.InputBootstrapper
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:582)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
... 6 more
** Hadoop 3.2.1 버전인 경우,
1. hadoop-3.2.1/share/hadoop/common/lib 경로에서 woodstox-core-5.0.3.jar 파일을 복사하여 Flume lib 폴더 하위에 복사해준다.
2. flume-env.ps1 또는 fluem2-env.sh 파일의 FLUME_CLASSPATH를 아래와 같이 잡아준다.
$FLUME_CLASSPATH="lib\hadoop-common-3.2.1.jar;lib\woodstox-core-5.0.3.jar"