搜索此博客

2016年12月31日星期六

内网穿透

原版
./sunny clientid 936abd06debec200

php版
php ./sunny.php --clientid=4ed377963456ee5a


op版
opkg install coreutils-nohup
nohup /ngrokc -SER[Shost:server.ngrok.cc,Sport:4443,Atoken:a045301f05a5b5f15402ea06fd0be0b8] -AddTun[Type:http,Lhost:127.0.0.1,Lport:80,Sdname:xuefliang]

nohup /ngrokc -SER[Shost:server.ngrok.cc,Sport:4443,Atoken:a045301f05a5b5f15402ea06fd0be0b8] -AddTun[Type:tcp,Lhost:127.0.0.1,Lport:22,Rport:63541]

python 版https://github.com/hauntek/python-ngrok/
添入
host = 'server.ngrok.cc' # Ngrok服务器地址
port = 4443 # 端口
bufsize = 1024 # 吞吐量

Tunnels = list() # 全局渠道赋值
body = dict()
body['protocol'] = 'http'
body['hostname'] = ''
body['subdomain'] = 'xuefengliang'
body['rport'] = 0
body['lhost'] = '127.0.0.1'
body['lport'] = 80
Tunnels.append(body) # 加入渠道队列

host和body['subdomain']
然后运行
python3 python-ngrok.py

路由器开机启动,exit 0加入
nohup /ngrokc -SER[Shost:server.ngrok.cc,Sport:4443,Atoken:a045301f05a5b5f15402ea06fd0be0b8] -AddTun[Type:http,Lhost:127.0.0.1,Lport:80,Sdname:xuefliang]  -AddTun[Type:tcp,Lhost:127.0.0.1,Lport:22,Rport:63541]

远程登录   ssh root@server.ngrok.cc -p 63541



2016年12月11日星期日

ubuntu install lamp


1 sudo apt install wget screen unzip -y
unzip lamp.zip
cd lamp-master/
chmod +x *.sh
3 screen -S lamp
sudo ./lamp.sh

R 连接 mysql

方式一
library(RMySQL)
con <- dbConnect(MySQL(),
                 user='root',
                 password='52332580',
                 dbname='douban',
                 host="127.0.0.1")
dbListTables(con)
# 设置编码格式
dbSendQuery(con, "SET NAMES utf8")
# 查询数据
query <-  "SELECT * FROM movie"
# 提取数据
data <-  dbGetQuery(con, query)

# 关闭连接
dbDisconnect(con)

方式二
library(tidyverse)
my_db <- src_mysql(dbname="douban",host = "127.0.0.1",port = 3306,user = "root",password = "52332580")
my_tbl <- tbl(my_db,"movie")
my_tbl%>%select(title,url)

乱码解决方式如下:
 1、sudo vim /etc/mysql/my.cnf
添加:
 [client]
default-character-set=utf8
2、sudo service mysqld restart

2016年12月10日星期六

2016年12月8日星期四

python 统计西游记字数

import json

fr = open("/home/xuefliang/PycharmProjects/Test/xyj.txt", "r")

characters = []
stat = {}

for line in fr:
   line = line.split()

   if len(line) == 0:
      continue
   # print type(line)
   temp = json.dumps(line, encoding="UTF-8", ensure_ascii=False)
   result = [t for t in temp][2:-2]

   for x in xrange(0, len(result)):
      if not result[x] in characters:
         characters.append(result[x])

      if not stat.has_key(result[x]):
         stat[result[x]]=0      stat[result[x]]+=1fr.close()

python中包含UTF-8编码中文的列表或字典的输出

num=[123]
b=repr(num)
c=[]

for x in xrange(1,len(b)-1):
    c.append(b[x])


s = u'字符串'
L= [c for c in s]

import json

char=[u'梁雪枫']
temp=json.dumps(char, encoding="UTF-8", ensure_ascii=False)
result=[t for t in temp][2:-2]

2016年12月5日星期一

linux mint ibus 拼音输入法

Linux Mint 加入 IBus libpinyin輸入法

安裝好 Linux Mint 時,


要在 Linux Mint 加入拼音輸入法, 參考步驟如下 :

1) 點擊 選單 > 偏好設定 > 輸入法 :


2) Linux Mint 有提供多種輸入法, 我們選擇安裝 IBus, 所以, 點擊 "為 IBus 加入支援" :


3) 之後, 會出現如下畫面, 再同樣位置, 點擊 "安裝選擇性組件" :

sudo apt install ibus-libpinyin

4) 選擇性組件安裝完畢後, 在輸入法選項中, 選擇 "IBus" :


5) 接下來的步驟, 就是要在 IBus 裡面加入注音輸入法, 點擊 選單 > 偏好設定 > iBus 偏好設定 :
(p.s. 若安裝好 IBus, 在偏好設定中卻找不到 iBus 偏好設定, 則建議重新啟動 Linux)


6) IBus 因為剛安裝完成, 尚未正式執行, 這時會出現如下畫面, 點擊 "Yes" 啟動 IBus :


7) 開啟 IBus Preferences 畫面後, 切換到 "Input Method", 然後點擊 "Add", 選擇 "漢語" :

Ubuntu下用ProxyChains穿墙安装Dropbox

1 sudo proxychains apt-get install nautilus-dropbox
2 proxychains dropbox start -i &

ubuntu 16 install R

1. Add R repository
First, we’ve got to add a line to our /etc/apt/sources.list file. This can be accomplished with the following. Note the “xenial” in the line, indicating Ubuntu 16.04. If you have a different version, just change that.
sudo echo "deb http://cran.rstudio.com/bin/linux/ubuntu xenial/" | sudo tee -a /etc/apt/sources.list
2. Add R to Ubuntu Keyring
First:
 gpg --keyserver keyserver.ubuntu.com --recv-key E084DAB9
Then:
 gpg -a --export E084DAB9 | sudo apt-key add -
3. Install R-Base
Most Linux users should be familiar with the old…
sudo apt-get update
sudo apt-get install r-base r-base-dev
4.
sudo apt-get install libfuse-dev libcurl4-openssl-dev libxml++2.6-dev libssl-dev openjdk-8-*  libglu1-mesa-dev  libgeos-dev libgdal1-dev libproj-dev libfreetype6-dev libgmp-dev libmpfr-dev

error: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

sudo apt install  python-dev

2016年12月4日星期日

2016年11月28日星期一

anaconda 卸载

1 、删除anaconda文件
2、修改.bashrc文件
export PATH="/home/xuefliang/anaconda2/bin:$PATH"
中的/home/xuefliang/anaconda2/bin:删除

2016年11月24日星期四

openwrt pcre 编译

1、下载prce的makefile,放在SDK中package中
https://github.com/openwrt/packages/tree/master/libs/pcre
2、make menuconfig选中shadowsocks.保存退出再执行make V=s 即可

2016年11月13日星期日

crouton使用

1、sudo sh ~/Downloads/crouton -r list
2 sudo sh ~/Downloads/crouton -r xenial -t core3 sudo enter-chroot4 sudo apt install vim git python bash-completion proxychains software-properties-common

sudo enter-chroot查看sudo edit-chroot -a删除sudo delete-chroot kde

crouton使用

1、sudo sh ~/Downloads/crouton -r list
2 sudo sh ~/Downloads/crouton -r xenial -t core3 sudo enter-chroot4 sudo apt install vim git python bash-completion proxychains software-properties-common

sudo enter-chroot查看sudo edit-chroot -a删除sudo delete-chroot kde

2016年11月10日星期四

raspberry树莓派安装CUPS实现打印服务器共享

1、sudo atp update
2、sudo apt install hplip vim
3、sudo vim /etc/cups/cupsd.conf

0.0.0.0:631
<Location />
Order allow,deny
Allow from @Local
</Location>
 
# Restrict access to the admin pages…
<Location /admin>
Order allow,deny
Allow from @Local
</Location>
 
# Restrict access to configuration files…
<Location /admin/conf>
AuthType Default
Require user @SYSTEM
Order allow,deny
Allow from@Local
</Location>
4、sudo service cups start

5、https://你的树莓派ip:631

6、添加http://你的树莓派ip:631/printers/打印机名

2016年11月8日星期二

Raspberry pi 远程

1、raspberry
sudo apt-get install xrdp
2、客户端
sudo apt-get install rdesktop 

rdesktop 192.168.10.117

2016年11月2日星期三

pandas 操作

# -*- coding: utf-8 -*-
import pandas as pd
data = {'Country': ['Belgium', 'India', 'Brazil'],
           'Capital': ['Brussels', 'New Delhi', 'Brasília'],
           'Population': [11190846, 1303171035, 207847528]}
df = pd.DataFrame(data, columns=['Country', 'Capital', 'Population'])

s = pd.Series([3, -5, 7, 4], index=['a', 'b', 'c', 'd'])

df['Country']

#By Position
# Select single value by row & column
df.iloc[0,0]
df.iat[0,0]

#By Label
#Select single value by row &  column labels
df.loc[0,'Country']
df.at[0, 'Country']

# By Label/Position
# Select single row of subset of rows
df.ix[2]
#Select a single column of  subset of columns
df.ix[:,'Capital']
# Select rows and columns
df.ix[1,'Capital']

#Boolean Indexing
# Series s   where value is not >1
s[~(s > 1)]
#s where value is <-1 or >2
s[(s < -1) | (s > 2)]
# Use filter to adjust DataFrame
df[df['Population']>1200000000]

#Dropping
#Drop values from rows (axis=0)
s.drop(['a', 'c'])
#Drop values from columns(axis=1)
df.drop('Country', axis=1)

#Sort & Rank
# Sort by row or column index
df.sort_index(by='Country')
# Sort a series by its values
s.order()
#Assign ranks to entries
df.rank()

#Retrieving Series/DataFrame Information
#(rows,columns)
df.shape
# Describe index
df.index
# Describe DataFrame columns
df.columns
#Info on DataFrame
df.info()
# Number of non-NA values
 df.count()

# Sum of values
df.sum()
# Cummulative sum of values
df.cumsum()
# Minimum/maximum values
df.min()/df.max()
#Minimum/Maximum index value
df.idxmax
df.idxmin
#Summary statistics
df.describe()
# Mean of values
df.mean()
#Median of values
df.median()

#Applying Functions
f = lambda x: x*2
df.apply(f)
df.applymap(f)

#Internal Data Alignment
s3 = pd.Series([7, -2, 3], index=['a', 'c', 'd'])
s + s3

#Arithmetic Operations with Fill Methods
s.add(s3, fill_value=0)
s.sub(s3, fill_value=2)
s.div(s3, fill_value=4)
s.mul(s3, fill_value=3)

#Read and Write to CSV
pd.read_csv('file.csv', header=None, nrows=5)
pd.to_csv('myDataFrame.csv')

#Read and Write to Excel
pd.read_excel('file.xlsx')
pd.to_excel('dir/myDataFrame.xlsx', sheet_name='Sheet1')
##Read multiple sheets from the same file
xlsx = pd.ExcelFile('file.xls')
df = pd.read_excel(xlsx, 'Sheet1')

#Read and Write to SQL Query or Database Table
from sqlalchemy import create_engine
engine = create_engine('sqlite:///:memory:')
pd.read_sql("SELECT * FROM my_table;", engine)
pd.read_sql_table('my_table', engine)
pd.read_sql_query("SELECT * FROM my_table;", engine)
pd.to_sql('myDf', engine)

2016年10月31日星期一

R 读取 utf-8编码的文件

Sys.setlocale(locale='Chinese')

library(readxl)
user <- read_excel("C:\\Users\\xuefliang\\Downloads\\vaccine_user_list.xls")

2016年10月27日星期四

Python on a chromebook

shell
sudo enter-chroot
sudo python shadowsocks/shadowsocks/local.py -c shadowsocks/user-config.json -d start

2016年10月19日星期三

firefox proxyswither 使用pac(白名单)

1、git clone https://github.com/breakwa11/gfw_whitelist

2、proxyswither Automatic中填写
file:/home/xuefliang/shadowsocks/shadowsocks/whitelist.pac

whitelist.pac中代理地址和端口和实际使用的需要一致


2016年10月18日星期二

python 数据预处理和pandas sklearn

import pandas as pd
import numpy as np
from sklearn import preprocessing,linear_model,metrics


data = pd.DataFrame({'pet': ['cat', 'dog', 'dog', 'fish', 'cat', 'dog', 'cat', 'fish'],
                    'children': [4., 6, 3, 3, 2, 3, 5, 4],
                    'salary': [90, 24, 44, 27, 32, 59, 36, 27]})
                   
lb = preprocessing.LabelBinarizer()
#标签二值化(Label binarization)
#LabelBinarizer通常用于通过一个多类标签(label)列表,创建一个label指示器矩阵
pdlb=pd.DataFrame(lb.fit_transform(data['pet']),columns=['cat','dog','fish'])
#横向合并
data=pd.concat([data,pdlb],axis=1)
#数据标准化(Standardization or Mean Removal and Variance Scaling)
#scale进行标准化缩放的数据均值为0,具有单位方差。
data['children']=preprocessing.scale(data['children'])
#数据规范化(Normalization)
#把数据集中的每个样本所有数值缩放到(-1,1)之间。
#X_normalized = preprocessing.normalize(X, norm='l2')
#二进制化(Binarization)
#将数值型数据转化为布尔型的二值数据,可以设置一个阈值(threshold)
#binarizer = preprocessing.Binarizer(threshold=1.1) # 设定阈值为1.1 


#scikit-learn要求X是一个特征矩阵,y是一个NumPy向量
#X可以是pandas的DataFrame,y可以是pandas的Series,scikit-learn可以理解这种结构
X=data[['children','salary','cat','dog','fish']]
y=data['salary']

mlr = linear_model.LinearRegression()
mlr.fit(X,y)

2016年10月13日星期四

非正态分布数据的集中趋势和离散趋势描述指标

a <- c(1,2,5,6,7)
#中位数
median(a)
#中位数绝对偏差(median absolute deviation)
mad(a)

mad()函数的结果默认乘了一个常数,将constant = 1,就是不乘常数。

MAD就是先求出给定数据的中位数(注意并非均值),然后原数列的每个值与这个中位数求出绝对差,然后再求新数列的中位数,就是MAD
例如 有数列85 9 6 3 2 3 4 9显然中位数是5,然后求出绝对差后的数列80 4 1 2 3 2 1 4所以MAD就是2.

实际上计算mad的过程就是:给定一个向量,先求出中位数,再求出原向量的每一个元素与该中位数的距离,从而得到一个新向量(元素全为大于零的数)。再求这个新向量的中位数。

cannot import name MLPClassifier

pip install scikit-learn==0.18.rc2
 
from sklearn.neural_network import MLPClassifier
X = [[0., 0.], [1., 1.]]
y = [0, 1]
clf = MLPClassifier(solver='lbgfs',alpha=1e-5,hidden_layer_sizes=(5, 2), random_state=1)
clf.fit(X, y) 
clf.predict([[2., 2.], [-1., -2.]])
参考文献:
http://scikit-learn.org/stable/modules/neural_networks_supervised.html 

2016年10月9日星期日

2016年9月17日星期六

vim 自动补全括号

sudo vim /usr/share/vim/vimrc

### sudo find / -name '*vimrc'

function! AutoPair(open, close)
        let line = getline('.')
        if col('.') > strlen(line) || line[col('.') - 1] == ' '
                return a:open.a:close."\<ESC>i"
        else
                return a:open
        endif
endf

function! ClosePair(char)
        if getline('.')[col('.') - 1] == a:char
                return "\<Right>"
        else
                return a:char
        endif
endf

function! SamePair(char)
        let line = getline('.')
        if col('.') > strlen(line) || line[col('.') - 1] == ' '
                return a:char.a:char."\<ESC>i"
        elseif line[col('.') - 1] == a:char
                return "\<Right>"
        else
                return a:char
        endif
endf

function! RemovePairs()
    let l:line = getline(".")
    let l:previous_char = l:line[col(".")-1]

    if index(["(", "[", "{"], l:previous_char) != -1
        let l:original_pos = getpos(".")
        execute "normal %"
        let l:new_pos = getpos(".")

        if l:original_pos == l:new_pos
            execute "normal! a\<BS>"
            return
        end

        let l:line2 = getline(".")
        if len(l:line2) == col(".")
            execute "normal! v%xa"
        else
            execute "normal! v%xi"
        end

    else
        execute "normal! a\<BS>"
    end
endfunction

function! RemoveNextDoubleChar(char)
    let l:line = getline(".")
    let l:next_char = l:line[col(".")]

    if a:char == l:next_char
        execute "normal! l"
    else
        execute "normal! i" . a:char . ""
    end
endfunction

inoremap ) <ESC>:call RemoveNextDoubleChar(')')<CR>a
inoremap ] <ESC>:call RemoveNextDoubleChar(']')<CR>a
inoremap } <ESC>:call RemoveNextDoubleChar('}')<CR>a
inoremap <BS> <ESC>:call RemovePairs()<CR>a
inoremap ( <c-r>=AutoPair('(', ')')<CR>
inoremap ) <c-r>=ClosePair(')')<CR>
inoremap { <c-r>=AutoPair('{', '}')<CR>
inoremap } <c-r>=ClosePair('}')<CR>
inoremap [ <c-r>=AutoPair('[', ']')<CR>
inoremap ] <c-r>=ClosePair(']')<CR>
inoremap " <c-r>=SamePair('"')<CR>
inoremap ' <c-r>=SamePair("'")<CR>
inoremap ` <c-r>=SamePair('`')<CR>

2016年9月9日星期五

R等比例抽样

library(caret)
#等比例抽样,按照物种分类的变量抽取10%的样本
splitindex <- createDataPartition(iris$Species,times = 1,p = 0.1,list = FALSE)
sample <- iris[splitindex,]

#亚变量处理
cor(iris[,1:4])
dmy <-dummyVars(~Species,data=iris)
result <- data.frame(predict(dmy,newdata = iris))

#Rattle
library(rattle)
rattle()

ubuntu 安装 flash 插件

sudo apt install flashplugin-installer

2016年9月6日星期二

forcats包简介

forcats

安装

# install.packages("devtools")
devtools::install_github("hadley/forcats")
## Skipping install of 'forcats' from a github remote, the SHA1 (004279de) has not changed since last install.
##   Use `force = TRUE` to force installation

使用

library(dplyr)
## 
## Attaching package: 'dplyr'
## The following objects are masked from 'package:stats':
## 
##     filter, lag
## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union
library(ggplot2)
library(forcats)
head(gss_cat)
## # A tibble: 6 × 9
##    year       marital   age   race        rincome            partyid
##   <int>        <fctr> <int> <fctr>         <fctr>             <fctr>
## 1  2000 Never married    26  White  $8000 to 9999       Ind,near rep
## 2  2000      Divorced    48  White  $8000 to 9999 Not str republican
## 3  2000       Widowed    67  White Not applicable        Independent
## 4  2000 Never married    39  White Not applicable       Ind,near rep
## 5  2000      Divorced    25  White Not applicable   Not str democrat
## 6  2000       Married    25  White $20000 - 24999    Strong democrat
## # ... with 3 more variables: relig <fctr>, denom <fctr>, tvhours <int>

改变因子的值(Change level values)

gss_cat %>% count(partyid)
## # A tibble: 10 × 2
##               partyid     n
##                <fctr> <int>
## 1           No answer   154
## 2          Don't know     1
## 3         Other party   393
## 4   Strong republican  2314
## 5  Not str republican  3032
## 6        Ind,near rep  1791
## 7         Independent  4119
## 8        Ind,near dem  2499
## 9    Not str democrat  3690
## 10    Strong democrat  3490
gss_cat %>%
  mutate(partyid = fct_recode(partyid,
    "Republican, strong"    = "Strong republican",
    "Republican, weak"      = "Not str republican",
    "Independent, near rep" = "Ind,near rep",
    "Independent, near dem" = "Ind,near dem",
    "Democrat, weak"        = "Not str democrat",
    "Democrat, strong"      = "Strong democrat"
  )) %>%
  count(partyid)
## # A tibble: 10 × 2
##                  partyid     n
##                   <fctr> <int>
## 1              No answer   154
## 2             Don't know     1
## 3            Other party   393
## 4     Republican, strong  2314
## 5       Republican, weak  3032
## 6  Independent, near rep  1791
## 7            Independent  4119
## 8  Independent, near dem  2499
## 9         Democrat, weak  3690
## 10      Democrat, strong  3490
gss_cat %>%
  mutate(partyid = fct_collapse(partyid,
    other = c("No answer", "Don't know", "Other party"),
    rep = c("Strong republican", "Not str republican"),
    ind = c("Ind,near rep", "Independent", "Ind,near dem"),
    dem = c("Not str democrat", "Strong democrat")
  )) %>%
  count(partyid)
## # A tibble: 4 × 2
##   partyid     n
##    <fctr> <int>
## 1   other   548
## 2     rep  5346
## 3     ind  8409
## 4     dem  7180

合并因子,保留最多的那n个因子

gss_cat %>% count(relig)
## # A tibble: 15 × 2
##                      relig     n
##                     <fctr> <int>
## 1                No answer    93
## 2               Don't know    15
## 3  Inter-nondenominational   109
## 4          Native american    23
## 5                Christian   689
## 6       Orthodox-christian    95
## 7             Moslem/islam   104
## 8            Other eastern    32
## 9                 Hinduism    71
## 10                Buddhism   147
## 11                   Other   224
## 12                    None  3523
## 13                  Jewish   388
## 14                Catholic  5124
## 15              Protestant 10846
#默认为1
gss_cat %>% 
  mutate(relig = fct_lump(relig)) %>% 
  count(relig)
## # A tibble: 2 × 2
##        relig     n
##       <fctr> <int>
## 1 Protestant 10846
## 2      Other 10637
gss_cat %>% 
  mutate(relig = fct_lump(relig, n = 5)) %>% 
  count(relig)
## # A tibble: 6 × 2
##        relig     n
##       <fctr> <int>
## 1  Christian   689
## 2       None  3523
## 3     Jewish   388
## 4   Catholic  5124
## 5 Protestant 10846
## 6      Other   913
#负数为反方向保留,prop为比例
gss_cat %>% 
  mutate(relig = fct_lump(relig, prop = -0.10)) %>% 
  count(relig)
## # A tibble: 12 × 2
##                      relig     n
##                     <fctr> <int>
## 1                No answer    93
## 2               Don't know    15
## 3  Inter-nondenominational   109
## 4          Native american    23
## 5                Christian   689
## 6       Orthodox-christian    95
## 7             Moslem/islam   104
## 8            Other eastern    32
## 9                 Hinduism    71
## 10                Buddhism   147
## 11                  Jewish   388
## 12                   Other 19717

Change order of levels:

fct_relevel(): move specified level up front.

fct_inorder(): order by first appearance of each level.

fct_reorder(): order by summary of another value (same as stats::reorder()).

fct_infreq():

order by frequency.

fct_shuffle(): randomly shuffle order of levels.

fct_rev(): reverse order of levels.

fct_shift(): shift levels to the left/right.

Change value of levels:

fct_anon(): anonymise factor levels.

fct_lump(): lump rarest (or most common) levels into “other”.

fct_recode(): manually recode levels.

Add new levels:

fct_expand(): add new levels to a factor.

fct_explicit_na(): turn missing values into an explicit factor.

A few other helpers:

fct_c(): concatenate factors using union of levels.

fct_count():

count occurences of levels, optionally sorting by frequency.

fct_unify(): ensure list of factors share the same levels.

fct_unique(): compute from levels of factor.

fct_drop(): drop levels without data (same as base::droplevels()).

lvls_union(): finds union of levels from list of factors.

relig <- gss_cat %>%
  group_by(relig) %>%
  summarise(
    age = mean(age, na.rm = TRUE),
    tvhours = mean(tvhours, na.rm = TRUE),
    n = n()
  )
ggplot(relig, aes(tvhours, relig)) + geom_point()

ggplot(relig, aes(tvhours, fct_reorder(relig, tvhours))) +
  geom_point()

by_age <- gss_cat %>%
  filter(!is.na(age)) %>%
  group_by(age, marital) %>%
  count() %>%
  mutate(prop = n / sum(n))

ggplot(by_age, aes(age, prop)) +
  geom_line(aes(colour = marital))