On the Research of CS

Here is a compliation of materials (mostly written) that once benefited me on my pursuit of a research career in computer science. Some are also collections of great resources,^[1]^[2] some are written by Turing award winners.^[3]^[4] While the selection of them associates with my personal perspective, I hope the readers could also benefit from them as I do.

# My collection

From a set of experts' views on NLP research.^[5]

Three biggest open problems in NLP (Jan 2019)

Grounded language learning
Language reasoning
Learning in low-resource settings (unsupervised, transfer, multitask, meta-learning, prior knowledge, etc)

Four suggestions to graduate students:

Read a lot to gain a strong background
Be ambitious and novel for the long-term result
Publish progress, even in a workshop for the short-term result
Spend about 10-20% time to learn to collaborate

From a series of articles subject to the aptitude of doing research written by Prof. Song-Chun Zhu.^[6]

There is undeniable a wide spread of utilitarianism among Chinese students and their parents, and it has become even stronger in the recent year since China has been through an economic revival. So it's more difficult and important for our Chinese students to overcome this obstacle. The only way out is to make it clear what you want and who you want to be. Some parents might explain that they just want their children to be happy and enjoy life, but as the saying goes, "the tree desires stillness but the wind will not cease". One needs to confront their fate.

Einstein once addressed a speech in Max Planck's sixtieth birthday^[7] in which he conveyed that there are three various motives leads people thither and dwell in the temple of science: some take to science out of a joyful sense of superior intellectual power; some others come for purely utilitarian purposes. Although they all contribute to the buildings of the temple of science, if there are only these two kinds of people, the temple would never have come to be, any more than a forest can grow which consists of nothing but creepers. Besides, there is the third kind of odd people who have a finely tempered nature longs to escape from the personal life into the world of objective perception and thought. Max Planck absolutely seat in the third group, so as many great scientists including Einstein himself. Whether the other people can become engineers, officers, tradesmen, or scientists depends on circumstances, but for the third kind of people, they're meant to be scientists.

Zhu gives two analogies about two traps researchers frequently step into. One is named street lamp of research. This story is described in a book written by Michael Arbib:

It's a dark night, you see a man looking for something right under the street lamp when you walk down the street. Then you ask him: "Are you certain about your key is lost here?" "No", he replies. And you go on asking: "So why are you keep looking the key here?" "I don't know, cause here is the only bright place, where else can I find my key?"

It might sound ridiculous, but it's mostly the case we are facing today when solving AI problems with deep learning.

The second analogy called double stampede event. It comes with a story Zhu personally experienced when he was a child. The main point is an unconscious crazy research trend can tear you apart.

A transcription of Dr. Richard Hamming's talk at a Bell lab seminar.^[3:1] It's a very special talk that is nothing about ordinary run-of-the-mill research, but great world-class research. As of where Hamming stands, he is among a few people who can carry on this kind of study and give some insights. Some points addressed in his speech does intrigue me, for example, the role luck, brain, and ambiguity about a theory play in the way to success.

Quoted from David Blackwell, "I've worked in so many areas – I'm sort of a dilettante. Basically, I'm not interested in doing research and I never have been. I'm interested in understanding, which is quite a different thing. And often to understand something you have to work it out yourself because no one else has done it."^[8]

Quoted from Andrej Karpathy, "You can’t expect to win in the long run by somehow gaming the system or putting up false appearances."^[9]

I found Sam Altman's advice on entrepreneurship also applies to CS research, especially as more (opens new window) and more (opens new window) research lab-like startups are being founded these days.^[10]

Usually people start off wanting to make a huge amount of money and end up wanting to create something important.
Always look for a project that, if successful, will make the rest of your career look like a footnote.
Trust the exponential, be patient, and be pleasantly surprised.
Have almost too much self-belief.
Come up with the most original ideas and find easy, fast ways to test them.
Selling what you truly belives in feels great, and trying to sell snake oil feels awful.
It’s easy—and human nature—to prioritize short-term gain and convenience over long-term fulfillment.
Find work you like doing with people you enjoy spending a lot of time with.
You get rich by owning things that grow rapidly in value, not salaries.
Be internally driven—to impress yourself rather than other people.

From a writing piece of Manuel Blum,^[4:1]

"Books are random access"
Write as you read (turning you from a finite automata to a Turing machine)
"... I could tell you something..."
"How SHOULD I have been thinking to solve that problem?"
"(Contradiction) is one of our most potent sources of knowledge."
"Make a list for yourself of good ways to pursue a problem.", e.g.,
- start thinking from "the middle of a (presumed) solution"
- try proving your belief wrong
"Brains are muscles. They grow strong with exercise."
Same as eyes, minds could have blind spots, which make things up for us.
"One does not have to be brilliant, a genius, to be special."

To see a world in a grain of sand
Or a heaven in a wild flower,
Hold infinity in the palm of your hand
And eternity in an hour.
William Blake (1757-1827)

The yet to be summarized, nevertheless, valuable materials.

^[11] ^[12] ^[13] ^[14] ^[15] ^[16] ^[17] ^[18] ^[19] ^[20] ^[21] ^[22] ^[23] ^[24] ^[25]

← How To Become A Front-end Developer On Paper Note-taking →