2012年10月31日 星期三

[mahout]run integration example


peicheng@TW-:~/code/mahout-distribution-0.7111/integration$ mvn jetty:run
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Building Mahout Integration
[INFO] task-segment: [jetty:run]
[INFO] ------------------------------------------------------------------------
[INFO] Preparing jetty:run
[INFO] [resources:resources {execution: default-resources}]
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 4 resources
[INFO] [compiler:compile {execution: default-compile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [resources:testResources {execution: default-testResources}]
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] [compiler:testCompile {execution: default-testCompile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [jetty:run {execution: default-cli}]
[INFO] Configuring Jetty for project: Mahout Integration
[INFO] Webapp source directory = /home/peicheng/code/mahout-distribution-0.7111/integration/src/main/webapp
[INFO] web.xml file = /home/peicheng/code/mahout-distribution-0.7111/integration/src/main/webapp/WEB-INF/web.xml
[INFO] Classes = /home/peicheng/code/mahout-distribution-0.7111/integration/target/classes
2012-10-31 10:33:54.999::INFO: Logging to STDERR via org.mortbay.log.StdErrLog
[INFO] Context path = /mahout-integration
[INFO] Tmp directory = determined at runtime
[INFO] Web defaults = org/mortbay/jetty/webapp/webdefault.xml
[INFO] Web overrides = none
[INFO] Webapp directory = /home/peicheng/code/mahout-distribution-0.7111/integration/src/main/webapp
[INFO] Starting jetty 6.1.10 ...
2012-10-31 10:33:54.075::INFO: jetty-6.1.10
2012-10-31 10:33:54.259::INFO: No Transaction manager found - if your webapp requires one, please configure one.
12/10/31 10:33:56 INFO file.FileDataModel: Creating FileDataModel for file /tmp/ratings.txt
12/10/31 10:33:56 INFO file.FileDataModel: Reading file info...
12/10/31 10:33:58 INFO file.FileDataModel: Processed 1000000 lines
12/10/31 10:33:58 INFO file.FileDataModel: Read lines: 1000209
12/10/31 10:33:58 INFO model.GenericDataModel: Processed 6040 users
12/10/31 10:33:58 INFO slopeone.MemoryDiffStorage: Building average diffs...
2012-10-31 10:34:31.983::INFO: Started SelectChannelConnector@0.0.0.0:8080
[INFO] Started Jetty Server


====================================


example URL
http://localhost:8080/mahout-integration/RecommenderServlet?userID=1&debug=true&howMany=100
http://localhost:8080/mahout-integration/RecommenderServlet?userID=1&howMany=100


2012年10月28日 星期日

[facebbook]graph api remove offline access token

[facebbook]graph api remove offline access token

移除了offline_accessc後,除了開始的兩個小時外,還可以另外延長到60天。

fbconsole/README.md at master · facebook/fbconsole
https://github.com/facebook/fbconsole/blob/master/README.md
Removal of offline_access permission - Facebook 開發人員
http://developers.facebook.com/roadmap/offline-access-removal/
Extending Access Tokens - Facebook 開發人員
http://developers.facebook.com/docs/howtos/login/extending-tokens/






2012年10月27日 星期六

Get long live access token from Facebook - Stack Overflow


Get long live access token from Facebook - Stack Overflow
http://stackoverflow.com/questions/10467272/get-long-live-access-token-from-facebook
offline_accessパーミッション廃止時の対応 : Facebook開発者向けドキュメントの日本語訳とTips
http://facebook-docs.oklahome.net/archives/52047013.html




2012年10月26日 星期五

[mahout]run apache mahout examples

run apache mahout examples

官方網站上面有個有個說明,

Recommender Documentation - Apache Mahout - Apache Software Foundation
https://cwiki.apache.org/confluence/display/MAHOUT/Recommender+Documentation

To build and run the demo, follow the instructions below, which are written for Unix-like operating systems:

  1. Obtain a copy of the Mahout distribution, either from SVN or as a downloaded archive.
  2. Download the "1 Million MovieLens Dataset" from Grouplens.org
  3. Unpack the archive and copy movies.dat and ratings.dat to trunk/integration/src/main/resources/org/apache/mahout/cf/taste/example/grouplens under the Mahout distribution directory.
  4. Navigate to the directory where you unpacked the Mahout distribution, and navigate to trunk.
  5. Run mvn -DskipTests install, which builds and installs Mahout core to your local repository
  6. cd integration
  7. You may need to give Maven more memory: in a bash shell, export MAVEN_OPTS=-Xmx1024M
  8. mvn jetty:run.
  9. Get recommendations by accessing the web application in your browser: http://localhost:8080/mahout-integration/RecommenderServlet?userID=1 This will produce a simple preference-item ID list which could be consumed by a client application. Get more useful human-readable output with the debug parameter: http://localhost:8080/mahout-integration/RecommenderServlet?userID=1&debug=true



遇到了一些問題

HTTP ERROR: 503

SERVICE_UNAVAILABLE

RequestURI=/mahout-integration/RecommenderServlet

Powered by jetty://





:~/code/mahout-distribution-0.7/integration$ mvn jetty:run
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Building Mahout Integration
[INFO] task-segment: [jetty:run]
[INFO] ------------------------------------------------------------------------
[INFO] Preparing jetty:run
[INFO] [resources:resources {execution: default-resources}]
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] [compiler:compile {execution: default-compile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [resources:testResources {execution: default-testResources}]
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] [compiler:testCompile {execution: default-testCompile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [jetty:run {execution: default-cli}]
[INFO] Configuring Jetty for project: Mahout Integration
[INFO] Webapp source directory = /home/peicheng/code/mahout-distribution-0.7/integration/src/main/webapp
[INFO] web.xml file = /home/peicheng/code/mahout-distribution-0.7/integration/src/main/webapp/WEB-INF/web.xml
[INFO] Classes = /home/peicheng/code/mahout-distribution-0.7/integration/target/classes
2012-10-26 14:58:11.752::INFO: Logging to STDERR via org.mortbay.log.StdErrLog
[INFO] Context path = /mahout-integration
[INFO] Tmp directory = determined at runtime
[INFO] Web defaults = org/mortbay/jetty/webapp/webdefault.xml
[INFO] Web overrides = none
[INFO] Webapp directory = /home/peicheng/code/mahout-distribution-0.7/integration/src/main/webapp
[INFO] Starting jetty 6.1.10 ...
2012-10-26 14:58:11.829::INFO: jetty-6.1.10
2012-10-26 14:58:12.973::INFO: No Transaction manager found - if your webapp requires one, please configure one.
2012-10-26 14:58:12.400::WARN: failed taste-recommender
javax.servlet.ServletException: java.lang.IllegalStateException: java.lang.ClassNotFoundException: org.apache.mahout.cf.taste.example.grouplens.GroupLensRecommender





在pom.xml內加入


${project.groupId}
mahout-examples
0.7


Problem running grouplens example
http://www.digipedia.pl/usenet/thread/10949/13891/
general - Re: Problem running grouplens example - msg#06145 - Recent Discussion OSDir.com
http://osdir.com/ml/general/2012-08/msg06145.html




2012年10月25日 星期四

(Title not set)







































































































   間



     程



8:30~9:00



報到



9:00~9:10



開幕暨致歡迎詞:東海大學  葉芳栢校長



9:10~10:10



專題演講:引導教育與大學公民願景:一位校友的奉獻與期待



主持人



主講人



羅文聰


東海大學學務長



吳兆田


alpha leader專業引導教師



10:10~10:30



茶敘



10:30~12:00



alpha體系的建立:制度整合經驗分享



主持人



分享人



回應人



范聖興


東海大學教務長



羅文聰


東海大學學務長



孫台鼎


靜宜大學學務長



12:00~13:00



午餐休息



13:00~14:00



引導教育在東海實錄分享



14:00~14:10



休息



14:10~15:40



引導教育融入課程與活動的經驗分享:教師成長歷程



主持人



分享人



回應人



羅文聰


東海大學學務長



伊志宗


東海大學


Alpha leader指導老師



藍美華


政治大學


政大書院執行長



15:40~15:50



休息



15:50~17:10



作為alpha leaders的學習意義:學生學習成效分享



主持人



分享人



回應人



王偉華


東海大學


博雅書院書院長



Alpha leaders



余佩珊


東海大學企管系副教授



17:10~17:20



休息



17:20~18:00



綜合討論


主持人:東海大學羅文聰學務長



18:00



賦歸



2012年10月16日 星期二

2012年10月15日 星期一

HBase

hbase介绍 - 阿里集团数据平台 alidata.org
http://www.alidata.org/archives/1509

about HFile
Th30z (Matteo Bertozzi Code): HBase I/O: HFile
http://th30z.blogspot.tw/2011/02/hbase-io-hfile.html
sstable入门阶段——HFile(version1)结构解析与c++读实现 | 伫立于架构,探究于细节
http://blog.hiwgy.com/2012/03/11/hfile_format_and_c_implement/

HADOOP中一种非典型两表JOIN的处理方法 - 阿里集团数据平台 alidata.org 一個partition問題~!!~
http://www.alidata.org/archives/204






2012年10月12日 星期五

[Mac]bashrc not work




By default, Terminal starts the shell via /usr/bin/login, which makes the shell a login shell. On every platform (not just Mac OS X) bash does not use .bashrc for login shells (only /etc/profile and the first of .bash_profile, .bash_login, .profile that exists and is readable). This is why “put source ~/.bashrc in your .bash_profile” is standard advice. –

[ubuntu]python easy install

How to install python easy_install tools for ubuntu

setuptools 0.6c11 : Python Package Index
http://pypi.python.org/pypi/setuptools



# apt-get install python-setuptools python-dev


2012年10月8日 星期一

Zookeeper 與 Hbase 為什麼要用zookeeper


Hbase 使用zookeeper 管理 HMaster 與 RegionServer。
Hbase也就是從BigTable來的,當table大到一個程度的時候,table會split成一個一個的region。
而RegionServer就把 HTable 存到Hadoop的HDFS。
Google BigTable使用Chubby來負責協調Node的工作,而HBase使用的就是zookeeper。

Trend Micro CDC SPN Team | Client HBase RegionServer
http://www.spnguru.com/tag/client-hbase-regionserver/
HBase技术介绍 « 搜索技术博客-淘宝
http://www.searchtb.com/2011/01/understanding-hbase.html




2012年10月2日 星期二

Hadoop in Taiwan 2012


很幸運上工沒多久,
就可以去參加這個活動。

早上六點半就出門,
七點半到辦公室一起搬螢幕跟電腦過去會場。

一如往常的大型的資訊活動都轉往中央研究院辦,
今年早上的演講大部分是被趨勢給包場了。

當然,還有聽到Steve 的喊話,
什麼是台灣人的新的機會呢?
我想就是腦力阿。

整個議程算是一個拋磚引玉,
越來越多應用用新的模式來呈現,
難怪乎當時有問cutting說,hadoop到底造成怎樣的影響,
他回答,出現了hadoop讓大家用新的方式思考Big Data,
真的,在hadoop出現以前,這些問題就早已經存在了。
只是這樣的pattern被更廣佈的炒熱了。