[solr] 使用 DataImportHandler 從 database 拉取資料並索引

sorl 的DataImportHandler 提供了batch 方式從database 拉取資料到 solr 裏面做索引

DataImportHandler - Solr Wiki

https://wiki.apache.org/solr/DataImportHandler#Delta-Import_Example

# cat data-config.xml

<dataConfig>

     <dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver" url="jdbc:mysql://host/database_name" user="username" password="password" batchSize="-1"/>

   <document>

       <entity name="myname" query="select id, uid, cname,utime from myinfo ORDER BY id" deltaImportQuery="select id, uid, cname,utime from myinfo where id='${dih.delta.id}' ORDER BY id" deltaQuery="select id from myinfo where utime > '${dataimporter.last_index_time}' ORDER BY id">

           <field column="id" name="id" />

           <field column="uid" name="uid" />

           <field column="cname" name="cname" />

           <field column="utime" name="utime" />

       </entity>

   </document>

</dataConfig>

cat dataimport.properties

#Fri Apr 21 13:08:08 UTC 2017

myname.last_index_time=2017-04-21 13\:05\:01

last_index_time=2017-04-21 13\:05\:01

可以在cron 內使用 curl 呼叫

0 * * * * curl

'http://localhost:8983/solr/userinfo/dataimport?command=delta-import&clean=false&commit=true&optimize=true' > /dev/null 2>&1