React-Router

Guide line in Chinese:

http://react-guide.github.io/react-router-cn/docs/guides/advanced/ConfirmingNavigation.html

https://cn.redux.js.org/docs/advanced/UsageWithReactRouter.html (use Redux with ReactRouter)

Guide line in English

https://reacttraining.com/react-router/web/guides/quick-start

https://redux.js.org/advanced/usage-with-react-router/ (use Redux with ReactRouter)

Redux

Action, Reducer,Store

https://cn.redux.js.org/docs/introduction/ (Chinese)

https://redux.js.org/basics/basic-tutorial (English)

Action: do some async job, like: ajax request, all the backend api. and then trigger a event to reducer

Reducer: listen to the event, and change the data in Store

Store: data store in it, and the components use connect + mapStageToProps to watch/compute the data change.

Read More

For What

When we developing a project and serving for the user with a lot of function. And some of this function is called in multiple place. when a new requirement is come which need to modify the code of the old function. How can you ensure the ensure old function won’t be broken by the change you make. human test? too native. Human always make mistake. So we need the machine to check for us.

Layer Test : Mockito

we use mockito to mock all the dependencies injected (the database, the third party api) to the layer, in the Controller layer , we mock the Service function, in the Service layer, we mock the Dao function;

Read More

What Selenide is?

Selenide is a framework for test automation powered by Selenium WebDriver that brings the following advantages:

Concise fluent API for tests. Powerful selectors. Simple configuration.

You don’t need to think how to shutdown browser, handle timeouts and StaleElement Exceptions or search for relevant log lines, debugging your tests.

Just focus on your business logic and let Selenide do the rest!

Seleium code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

public class GoogleTest {


chromeDriver webdriver;



@Before
public void setUp() {
System.setProperty("webdriver.chrome.driver","drivers/chromedriver.exe");

WebDriverManager.chromedriver().setup();

webdriver = new ChromeDriver();

}


@Test
public void searchGoogle() {
webdriver.get("[https://www.google.com/ncr](https://www.google.com/ncr)");
WebElement q = webdriver.findElementByName("q");
q.sendKeys("test");
q.submit();
wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("resultStats")));

}



@After
public void tearDown() {
if (webdriver != null) {
webdriver.quit();
}

}



}

Selenide Code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27


@Listeners(BrowserPerClass.class)
public class GoogleTest {


@Before
public void setUp() {
System.setProperty("webdriver.chrome.driver", "drivers/chromedriver.exe");
Configuration.browser = WebDriverRunner.CHROME;
Configuration.baseUrl = "[http://google.com](http://google.com/)";
}



@Test
public void searchGoogle() {
open("/");
$(By.name("q")).sendKeys("test");
$(By.name("q")).submit();
$(By.id("resultStats")).waitUntil(exist, 10000);

}



}

get element using Xpath, cssSelector

some element have no id or name, is not easy to locate it . there is two ways to locate it: Xpath, cssSelector. but how to know what the element’s xpath/cssSelector is? Chrome!!

open the page in Chrome and press F12 to open the Developer Tool Panel. Select the element and right click to copy the xpath/cssSelector. something like:

//*[@id=”tsf”]/div[2]/div/div[3]/center/input[1]

RUN ln -sf /dev/stdout /var/log/nginx/access.log

mount bind space to collect log

--mount type=bind,src=/opt/logs, dst=/usr/local/tomcat/logs/

mount volume to collect log

--mount type=volume src=volume_name dst=/use/local/tomcat/logs/

use redis/mq to collect log

docker→ redis/mq→ logstash → elasticsearch

Now

all the bugfix are commit direct to ‘bugfix’ branch, all members commit in the same branch. it make the commit history unreadable and everyone have to pull&merge others’s code which is not the current work in their hand frequently and make git hard to handle.

How to change

Use Github Flow

git flow guide

  1. Update master to latest upstream code
  2. Create a feature branch from master branch git checkout -b feature/myFeatureBranch
  3. Do the feature/work
  4. Push feature branch to origin
  5. Create pull request from origin/ -> upstream/master
  6. Review, fix raised comments, merge your PR or even better, get someone else to.

The main rule of GitHub Flow is that master should always be deployable.

Automatic Deploy For Test

  1. set github project hook
  2. commit pr to develop branch, tigger github project hook
  3. the hook send request to an url to the deploy server: jenkin or program you build.
  4. the deploy server pull the newest code and restart server
  5. test!

Read More

什么是事务

请先了解事务相关的概念。事务

Please learn about transaction first.Database_transaction

Mongodb的事务

MongoDb 4.0版本之前,没有事务。 如何在没有事务的情况下,使用mongoDb?

There is no transaction in MongoDb before version 4.0. How to deal with it?

MongoDb中,一个写操作在一个document上是原子的。即使是操作这个document的嵌套的子document。

因为没有事务,当你更新多个文档(documents)的时候,一些错误发生了,mongoDb不会进行rollback处理。 或者是多个操作时,个个操作会有交错,导致数据更新错误(如:多个操作更新用户资产)。

In MongoDb, a write operation is atomic on the level of a single document, even if the operation modifies multiple embedded documents within a single document.

Because there is no transaction, when you need to update multiple documents. if there is something error, it won’t rollback by itself。

如何处理这些情况呢?

Read More

MongoDb 非关系数据库 NoSql 数据库一员, 将数据使用灵活,类JSON形式进行存储。更多:what-is-mongodb

MongoDb is a member of the NoSql, it stores data in flexible, JSON-like documents.

概念 Concepts between SQL and MongoDb

SQL MongoDb
Table Collection
Row Document
Column Field
JOINs Embedded documents, $lookup & $graphLookup
GROUP_BY Aggregation -〉group by

操作 Operation between SQL and MongoDb

SQL MongoDb
INSERT INTO users (user_id, age, status) VALUES ‘bcd001’, 45, ‘A’) db.users.insert({ user_id: ‘bcd001’, age: 45, status: ‘A’})
SELECT * FROM users db.users.find()
UPDATE users SET status = ‘C’ WHERE age > 25 db.users.update( { age: { $gt: 25 } }, { $set: { tatus: ‘C’ } }, { multi: true })
db.start_transaction() cursor.execute(orderInsert, orderData) cursor.execute(stockUpdate, stockData)

db.commit()|s.start_transaction() orders.insert_one(order, session=s) stock.update_one(item, stockUpdate, session=s)s.commit_transaction() (new feature in 4.0 version)|

最大区别 Biggest different between SQL and MongoDb

SQL MongoDb
使用关系型数据库, 需要在数据库中定义对应的表和字段关联。当你需要修改到对应的字段, 必须对数据库进行停机修改,会影响现有的所有数据, 降低应用可用性。In SQL, you pre-define your database schema based on your requirements and set up rules to govern the relationships between fields in your tables.Any changes in schema necessitates a migration procedure that can take the database offline or significantly reduce application performance. 使用MongoDb, 字段 无需在数据库中进行定义,使用代码定义对应的关系。如果需要添加一个新的字段,无需影响其他的document,无需停机进行更新。In MongoDb, Fields is no need to declare the structure of documents to the system – documents are self-describing in Code。If a new field needs to be added to a document,then the field can be created without affecting all other documents in the collection,without updating a central system catalog, and without taking the system offline.
更多的控制在数据库(DBA), 索引,字段 都需要在数据库进行定义处理More control in Db, Indexes. Filed all controlled in Db side. 更多的控制在代码(开发), 开发可以在代码中定义对应的索引,字段结构。Less control in Db, more control in Code. Indexes, Fields can controlled in Code side.

参考:mongodb-mysql

Select 只获取必要的字段

why:

select *会增加cpu/io/内存/带宽的消耗

指定字段能有效利用索引覆盖:find({id: “test”}).select({id: 1}) 命中索引id, 因只select了id, 所以mongo直接返回了id, 不会再索引到具体的document,效率大大提高。

避免过度使用嵌入文档,最多只能一层, 不使用数组嵌套

why:

document有16MB限制。

尽量避免使用负向查询以及%开头的模糊查询

why:

ne、nin %开头 无法有效使用索引

禁止使用look up

why:

分片后,look up操作失效

不在数据库中进行数据计算

why:

js精度问题 (0.1 + 0.2 = 0.30000000000000004__)

cpu计算导致数据库卡顿,解放数据库CPU,把复杂逻辑计算放到服务层。

平衡范式与冗余

为提高效率,可以冗余数据

拒绝大sql,大批量 操作

禁止在更新十分频繁、区分度不高的属性上建立索引

why:

更新会变更B+树,更新频繁的字段建立索引会大大降低数据库性能

“性别”这种区分度不大的属性,建立索引是没有什么意义的,不能有效过滤数据,性能与全表扫描类似

索引内存也是有限的

使用explain 优化查询

查看explain结果,

可以简单查看inputStages.stage判断查询效率。

COLLECTION:全表查询

IXSCAN:命中索引查询

禁止使用自带全文索引

why:

效率低

MongoDb_Demo

参考: mysql 50条军规

co流程控制

nodejs常见异步流程

使用回调

1
2
3
4

fs.readFile("./file.js",function(err,data){
console.log(data.toString());
})

使用promise

1
2
3
4
5
6
7

var Promise = require("bluebird")
var fs = require("fs");
fs = Promise.promisifyAll(fs);
fs.readFileAsync("./test.js").then(function(data){
console.log(data.toString());
})

使用co+promise

1
2
3
4
5
6
7
8

var fs = require("fs");
var Promise = require("bluebird")
fs = Promise.promisifyAll(fs);
co(function* (){
var data = yield fs.readFileAsync('./test.js');
console.log(data.toString());
})

结论:使用co模块编写代码,形式与同步代码一致,没有回调和then,逻辑更加清晰。

Read More

#mongodb3.* 设置用户认证

1、mongodb是没有默认管理员账号,所以要先添加管理员账号,在开启权限认证。
2、切换到admin数据库,添加的账号才是管理员账号。
3、用户只能在用户所在数据库登录,包括管理员账号。
4、管理员可以管理所有数据库,但是不能直接管理其他数据库,要先在admin数据库认证后才可以。这一点比较怪

添加管理员账号:
默认情况下系统中没有用户,正常启动mongo:mongod

Read More