Bigdata, Hadoop ecosystem, Semantic IoT등의 프로젝트를 진행중에 습득한 내용을 정리하는 곳입니다.
필요한 분을 위해서 공개하고 있습니다.
문의사항은 gooper@gooper.com로 메일을
보내주세요.
오류내용#1처럼 fuseki endpoint를 통하여 요청을 보낼때 순간적으로 많은 요청이 발생하면 많은양의 소켓케넥션이 TIME-WAIT상태(netstat -a | grep 사용포트(예,23030))가 되면서 오류가 발생할 수 있는데 이럴때는
echo "1" > /proc/sys/net/ipv4/tcp_tw_reuse
and/or
echo "1" > /proc/sys/net/ipv4/tcp_tw_recycle
를 수행하여 TIME-WAIT상태의 소켓 커넥션을 재사용할 수 있도록 설정해준다.
이렇게 처리해주면 오류내용#2가 발생하는데 이는 커넥션 재사용에 커넥션은 형성되었는데 이를 jena가 처리하지 못하는 과부하 상태를 의미한다.
* 참고 : http://stackoverflow.com/questions/1572215/how-to-avoid-a-noroutetohostexception
-------------------------------오류내용#1-------------------------------------------
[2016-10-12 13:56:57,031] [service.llluminationStateJobService] [execute(#118)] [DEBUG] Exception ....................................> HttpException: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다: Unexpected error making the query: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다
[2016-10-12 13:56:57,031] [service.LectureTimeOnStateJobService] [execute(#112)] [DEBUG] Exception ....................................> HttpException: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다: Unexpected error making the query: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다
[2016-10-12 13:56:57,031] [service.ElectricPowerStateJobService] [execute(#115)] [DEBUG] Exception ....................................> HttpException: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다: Unexpected error making the query: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다
[2016-10-12 13:56:57,032] [service.HumanExistStateJobService] [execute(#116)] [DEBUG] Exception ....................................> HttpException: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다: Unexpected error making the query: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다
[2016-10-12 13:56:57,032] [service.ClimateStateJobService] [execute(#115)] [DEBUG] Exception ....................................> HttpException: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다: Unexpected error making the query: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다
[2016-10-12 13:56:57,034] [service.ProjectorStateJobService] [execute(#115)] [DEBUG] Exception ....................................> HttpException: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다: Unexpected error making the query: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다
[2016-10-12 13:56:57,034] [service.LectureTimeOffStateJobService] [execute(#112)] [DEBUG] Exception ....................................> HttpException: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다: Unexpected error making the query: java.net.NoRouteToHostException: 요청한 주소를 배정할 수 없습니다
-------------------------------오류내용#2-----------------------------------------
[2016-10-13 10:44:24,487] [sf.SparqlService] [updateSparql(#376)] [DEBUG] delete->insert sparql start............................
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#138)] [DEBUG] count of idxVals : 2
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#139)] [DEBUG] values of idxVals : [http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data, http://www.iotoasis.org/herit-in/herit-cse/TempSens
or_LR0001TS0005/status/Data/CONTENT_INST_2409764]
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#146)] [DEBUG] sparql to make ===========>
prefix o: <http://www.iotoasis.org/ontology/> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> delete { <@{arg0}> o:hasLatestContentInstance ?o . } WHERE { <@{arg0}> o:hasLatestContentInstance ?o .}
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#268)] [DEBUG] sparql made ===========>
prefix o: <http://www.iotoasis.org/ontology/> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> delete { <http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data> o:hasLatestContentInstance
?o . } WHERE { <http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data> o:hasLatestContentInstance ?o .}
org.apache.jena.atlas.web.HttpException: java.net.ConnectException: 연결이 거부됨
at org.apache.jena.riot.web.HttpOp.exec(HttpOp.java:1202)
at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:803)
at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:578)
at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:534)
at org.apache.jena.sparql.modify.UpdateProcessRemote.execute(UpdateProcessRemote.java:79)
at com.gooper.icbms.sda.sf.SparqlService.runModifySparql(SparqlService.java:404)
at com.gooper.icbms.sda.sf.SparqlService.updateSparql(SparqlService.java:378)
at com.gooper.icbms.sda.sf.TripleService.addLatestContentInstance(TripleService.java:173)
at com.gooper.icbms.sda.kafka.onem2m.AvroOneM2MDataSubscribe$1.run(AvroOneM2MDataSubscribe.java:116)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: 연결이 거부됨
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:120)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:179)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:328)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:612)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:447)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:884)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.jena.riot.web.HttpOp.exec(HttpOp.java:1186)
6182859,25-32 99%
[2016-10-13 10:44:24,487] [sf.TripleService] [addLatestContentInstance(#172)] [DEBUG] this.getInstanceUri() =====>http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data/C
ONTENT_INST_2409764
[2016-10-13 10:44:24,487] [sf.SparqlService] [updateSparql(#376)] [DEBUG] delete->insert sparql start............................
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#138)] [DEBUG] count of idxVals : 2
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#139)] [DEBUG] values of idxVals : [http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data, http://www.iotoasis.o
rg/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data/CONTENT_INST_2409764]
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#146)] [DEBUG] sparql to make ===========>
prefix o: <http://www.iotoasis.org/ontology/> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> delete { <@{arg0}> o:hasLatestContentInstance ?o . } WHERE { <@{arg0}> o:hasLatestC
ontentInstance ?o .}
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#268)] [DEBUG] sparql made ===========>
prefix o: <http://www.iotoasis.org/ontology/> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> delete { <http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Da
ta> o:hasLatestContentInstance ?o . } WHERE { <http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data> o:hasLatestContentInstance ?o .}
org.apache.jena.atlas.web.HttpException: java.net.ConnectException: 연결이 거부됨
at org.apache.jena.riot.web.HttpOp.exec(HttpOp.java:1202)
at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:803)
at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:578)
at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:534)
at org.apache.jena.sparql.modify.UpdateProcessRemote.execute(UpdateProcessRemote.java:79)
at com.gooper.icbms.sda.sf.SparqlService.runModifySparql(SparqlService.java:404)
at com.gooper.icbms.sda.sf.SparqlService.updateSparql(SparqlService.java:378)
at com.gooper.icbms.sda.sf.TripleService.addLatestContentInstance(TripleService.java:173)
at com.gooper.icbms.sda.kafka.onem2m.AvroOneM2MDataSubscribe$1.run(AvroOneM2MDataSubscribe.java:116)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: 연결이 거부됨
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:120)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:179)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:328)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:612)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:447)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:884)
6182859,25-32 99%
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:120)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:179)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:328)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:612)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:447)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:884)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.jena.riot.web.HttpOp.exec(HttpOp.java:1186)
... 11 more
[2016-10-13 10:44:24,470] [sf.SparqlService] [runModifySparql(#407)] [DEBUG] Exception message in runModifySparql() =====> java.net.ConnectException: 연결이 거부됨
[2016-10-13 10:44:24,470] [util.Utils] [restartFuseki(#494)] [DEBUG] fuseki-server restart start ...................
[2016-10-13 10:44:24,470] [util.Utils] [killFuseki(#416)] [DEBUG] killFuseki start==========================>
[2016-10-13 10:44:24,487] [onem2m.AvroOneM2MDataSubscribe$1] [run(#95)] [DEBUG] count data from kafka broker : 39995
[2016-10-13 10:44:24,487] [sf.TripleService] [getTriple(#73)] [DEBUG] === ri : CONTENT_INST_2409764, ty : 4 ====>include
contentType : normal
[2016-10-13 10:44:24,487] [onem2m.AvroOneM2MDataSubscribe$1] [run(#115)] [DEBUG] delete->insert o:hasLatestContentInstance start....................
[2016-10-13 10:44:24,487] [sf.TripleService] [addLatestContentInstance(#171)] [DEBUG] this.getParentResourceUri() ====> http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data
[2016-10-13 10:44:24,487] [sf.TripleService] [addLatestContentInstance(#172)] [DEBUG] this.getInstanceUri() =====>http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data/CONTENT_INST_2409764
[2016-10-13 10:44:24,487] [sf.SparqlService] [updateSparql(#376)] [DEBUG] delete->insert sparql start............................
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#138)] [DEBUG] count of idxVals : 2
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#139)] [DEBUG] values of idxVals : [http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data, http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data/CONT
ENT_INST_2409764]
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#146)] [DEBUG] sparql to make ===========>
prefix o: <http://www.iotoasis.org/ontology/> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> delete { <@{arg0}> o:hasLatestContentInstance ?o . } WHERE { <@{arg0}> o:hasLatestContentInstance ?o .}
[2016-10-13 10:44:24,487] [sf.SparqlService] [makeSparql(#268)] [DEBUG] sparql made ===========>
prefix o: <http://www.iotoasis.org/ontology/> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> delete { <http://www.iotoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data> o:hasLatestContentInstance ?o . } WHERE { <http://www.i
otoasis.org/herit-in/herit-cse/TempSensor_LR0001TS0005/status/Data> o:hasLatestContentInstance ?o .}
org.apache.jena.atlas.web.HttpException: java.net.ConnectException: 연결이 거부됨
at org.apache.jena.riot.web.HttpOp.exec(HttpOp.java:1202)
at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:803)
at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:578)
at org.apache.jena.riot.web.HttpOp.execHttpPost(HttpOp.java:534)
at org.apache.jena.sparql.modify.UpdateProcessRemote.execute(UpdateProcessRemote.java:79)
at com.gooper.icbms.sda.sf.SparqlService.runModifySparql(SparqlService.java:404)
at com.gooper.icbms.sda.sf.SparqlService.updateSparql(SparqlService.java:378)
at com.gooper.icbms.sda.sf.TripleService.addLatestContentInstance(TripleService.java:173)
at com.gooper.icbms.sda.kafka.onem2m.AvroOneM2MDataSubscribe$1.run(AvroOneM2MDataSubscribe.java:116)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: 연결이 거부됨
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:120)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:179)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:328)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:612)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:447)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:884)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.jena.riot.web.HttpOp.exec(HttpOp.java:1186)
... 11 more