[Reader-list] First Posting -- Peer-to-peer News distribution network

Sat Jan 24 23:37:33 IST 2004

Hello,
	We are working on implementing a Peer-to-Peer News 
Distribution Network which we believe will be the first of its kind and will 
usher in a new paradigm in the field of news distribution. A short write-
up discussing the background and motivation, the features needed to 
be implemented, etc. is given below.

Current News Distribution Scenario.
----------------------------------------------------
Traditionally news distribution has been the preserve of the rich and 
powerful media corporations. To run a newspaper, a television channel 
or a radio station needs so much expertise, capital investment and 
management skills that it is not possible for any individual or small 
group of individuals to engage in distribution of news. It is true that 
India (especially West Bengal) has a culture of publishing what is called 
"little magazines" but they have a limitation of limited circulation and are 
usually for publishing literary works as opposed to presentation of news 
items to the general public at large on a regular basis.

The individual could not disseminate news of his/her choice using the 
traditional news distribution channels as the individual is totally 
dependent on the large media corporations for news distribution. These 
corporations have their own agenda and limitations besides being 
subjected to censorship whether self-imposed or externally dictated.

Internet was a major break through in the area of news distribution. For 
the first, the ordinary people found an avenue to publish their views and 
also any news items of their choice. However to publish such news 
items an individual needs a web space which usually costs a significant 
amount of money. Also, these websites usually operate in logical 
isolation or at the maximum as a member of some web-ring. Thus there 
is no general facility to search for news items on such web sites other 
than the normal web search facilities provided by companies like 
Google. This is a great drawback of the Internet based news publication 
making it limited in its success as effective mass media. Also, the 
websites suffer from problems associated with centralised control like 
censorship and denial of service. Besides, the load of distributing the 
news is borne mainly by the publisher of the news item and not shared 
by the users. This results in high resource requirements and 
consequently large investments in case of sites that are frequently 
visited. Though there are plenty of small websites publishing news 
items run by individuals, these sites suffer from poor visibility and thus 
do not achieve the goal of providing an effective alternative news 
distribution channel. Thus we find that the Internet based news 
distribution is controlled largely by the same media corporations that 
control the traditional media channels.

We believe that a peer-to-peer news distribution network can overcome 
some the major short-comings of the present news distribution 
channels.

Brief Introduction to Peer-to-Peer Computing.
------------------------------------------------------------
One of the oldest uses of the term "peer-to-peer computing" is in IBM's 
Systems Network Architecture documents on LU6.2 Transactions, over 
25 years ago. The term, which we shall use interchangeably with p2p, 
came to the fore very publicly with the rise and fall of Napster [7]. 
Peer-to-peer systems are distributed Internet applications in which the 
resources of a large number of autonomous participants are harnessed 
in order to carry out the system's function. In many cases, peers form 
self-organising networks that are layered over the top of conventional 
Internet protocols and have no centralized structure. 

Peer-to-peer systems have evolved to support resource sharing in an 
environment characterised by users potentially numbering millions, 
most with homogeneous desktop systems and low bandwidth, 
intermittent connections to the Internet. As such, the emphasis has 
been on global fault-tolerance and massive scalability.

Features Desired in a Peer-to-peer News Distribution Network.
----------------------------------------------------------------------------------
o	Empowerment: Any user of the network should be empowered 
to distribute news items on any topic should he/she desire to do so. 
This is the primary requirement of the news distribution network.

o	Indic Script Support: We should be able to support Indic scripts 
natively (maybe using Unicode) so that the news items in Indian 
languages can be distributed and viewed seamlessly. We consider this 
feature essential since most of the Indian users are not native English 
speakers and the network will fail to fulfil its objective of empowerment if 
it remains restricted to the English-speaking elite population. 

We request any reader with expertise in Indic support to offer his/her 
help to us. 

o	Low Bandwidth Requirements: Most of the users, especially in 
India, are expected to have intermittent and low-bandwidth connection 
to the Internet. Thus the news distribution network should be such that 
the users can use the network fruitfully. This would require careful 
design of the network so as to incorporate feature such as on-the-fly 
compression/decompression, resumption of interrupted transfers, file-
splitting (so as to minimise the load on a single peer), and intelligent 
routing substrate. 

o	Multimedia Support: The network and the associated software 
should be able to handle multimedia news items like video clips and 
audio recordings. The impact and importance of the electronic visual 
media cannot be overlooked while creating a system for the future. 
Also, multimedia support gains greater significance in the Indian context 
since a large portion of the population is illiterate or more importantly 
semi-literate (i.e. literate only for the purpose of government statistics) 
and audio/video news items will be more accessible for them. These 
people can always ask someone to search and locate the item but 
would not need anyone to read the news to them.

o	Efficient Search: The network should support fast, efficient and 
effective search facilities. The user should be able to search for news 
items using any of a large number of criteria like topic, date, origin, 
locality and author/publisher. Providing such a search facility will be a 
real challenge since we will have to deal with Indic scripts and 
multimedia news items. Also, the search should not generate significant 
traffic on the Internet so as not to consume a huge amount of 
bandwidth.

o	Permanence of News Items: News items once published for 
distribution should remain on the network and should be accessible for 
a considerable period of time after which they may be stored in archival 
storage from where it can be retrieved if required. However, any request 
by the original publisher should be honoured by the peers.

o	Immutability: The network should ensure the integrity of the 
news articles submitted for distribution and any attempts to modify the 
news item (except by the author/publisher) should be resisted. Any user 
should be able to verify the integrity of a particular document if he/she 
wishes to do so. But this feature should not prevent any user to 
comment on an article, etc. and such comments may be viewed by 
other users. Adobe PDF documents support such comments.

o	Authentication: Any author/publisher may add his/her digital 
signature to a news item and other users should be able to verify such 
signatures in order to confirm the origin of the article/news item. This 
feature can be easily incorporated using asymmetric key cryptography 
techniques.

o	Anonymity and Refutability: By masking the identity of sources 
and sinks of requests, the network should be able to protect users from 
the potential censor or unsavoury agency. However we need to mask 
location information as well as identifiers, as otherwise a traffic analysis 
may effectively reveal identities, so some form of onion routing is also 
usually required.

The process of news item submission should be such that it would not 
be possible to track down the submitter unless the submitter purposely 
reveals his/her origin.

This two related features would make it very difficult to allow deletion or 
modification of news items even by the original submitter. So we need 
to find a way to balance these conflicting requirements.

o	Censor-Resistant: The network should be so designed as to 
withstand any attempts at censorship or Denial of Service. This may be 
achieved by splitting a file into component parts so that no single site 
carries the whole file, and a Denial of Service attack has to run over 
multiple sites.

Encrypting blocks of the file not only make sure that disclosure is 
unlikely, but also that a node can deny knowledge of the actual content 
it carries. This would help Service Providers to align themselves with 
the ``common carrier'' defence against legal liability for content, like 
telephony and postal providers.

o	Fully De-centralised: There should not be any centralised 
control so as to make the system fault-tolerant and efficient and to 
eliminate a single point of failure for legal, economic and political 
attacks. But this would make routing and searching more difficult. 

o	Platform Independent: The users should be able to access the 
news distribution network from any platform. So we need to create 
clients to access the network for the major platforms. In this respect we 
feel Java might be a good choice as a programming language.

o	Efficient Routing: Efficient routing is very essential in such a 
massively distributed system with no central control. The routing 
protocol should not generate much traffic for each routing request, 
otherwise the network will not be very usable. We intend to use a third-
generation routing substrate based on Distributed Hash Table for 
routing.

o	Support for trust models: Just as in the real world, a fully 
cooperative model such as this network may break down if peers are 
not provided with incentives to participate. We would look into trust, 
incentive and reputation models based on both economic and game 
theoretic approaches. This is one area where we envision the need for 
extensive research as there is no established model for trust and 
reputation.

o	Load Sharing: Most of the users will be using dial-up 
connections to the Internet. So it is imperative that users who have 
higher bandwidth share a larger portion of the load. Our network will 
have to facilitate such means-based load sharing.

The peer-to-peer news distribution network which we propose to 
develop will have a considerable impact on the way news is currently 
distributed. It will also be a pioneer in providing native Indic script 
support allowing the non English speaking people to benefit from this 
effort. A peer-to-peer computing experiment of this type has never been 
attempted in India. This will benefit the peer-to-peer networking 
community by allowing it to experiment with various new algorithms and 
techniques.

However, as you would appreciate, implementing all the features 
mentioned above is a Herculean task and is just not feasible in a time-
frame of just 6 months. 

So we would like the readers to suggest which features should be given 
priority.

Hope we would get the advice and support of the readers of this list. 
We would especially like to hear from the readers with backgrounds in 
journalism, etc. since the program is basically a vehicle to distribute 
news items.

Thank you.
Soumava Das
Sarai Independent Fellow

References

1.	ANDERSON, R. J. 
The Eternity Service, June 1997. 
http://www.cl.cam.ac.uk/users/rja14/eternity/ eternity.html. 

2.	ASPNES, J., DIAMADI, Z., AND SHAH, G. 
Fault-tolerant routing in peer-to-peer systems. 
In Twenty-First ACM Symposium on Principles of Distributed Computing 
(Monterey, USA, July 2002), pp. 223-232. 

3.	BLOOM, B. H. 
Space/time tradeoffs in hash coding with allowable errors. 
Comm. of the ACM 13, 7 (July 1970), 422. 10 

4.	CLARKE, I., SANDBERG, O., WILEY, B., AND HONG, T. W. 
Freenet: A distributed anonymous information storage and retrieval 
system. 
Lecture Notes in Computer Science 2009 (2001). 

5.	FREEDMAN, M. J., AND VINGRALEK, R. 
Efficient peer-to-peer lookup based on a distributed trie. 
In Proceedings of the 1st International Workshop on Peer-to-Peer 
Systems (IPTPS02) (Cambridge, MA, March 2002). 

6.	MORETON, T. D., PRATT, I. A., AND HARRIS, T. L. 
Storage, Mutability and Naming in Pasta. 
In Proceedings of the International Workshop on Peer-to-Peer 
Computing at Networking 2002, Pisa, Italy. (May 2002). 

7.	NAPSTER. 
Napster media sharing system. 
http://www.napster.com/. 

8.	RIPEANU, M. 
Peer-to-peer architecture case study: Gnutella network. 
In 2001 International conference on P2P computing (August 2001). 
http://people.cs.uchicago.edu/~matei/PAPERS/ P2P2001.pdf. 

9.	ROWSTRON, A., AND DRUSCHEL, P. 
Pastry: Scalable, decentralized object location, and routing for large-
scale peer-to-peer systems. 
Lecture Notes in Computer Science 2218 (2001), 329-350. 

10.	SUN. 
Jxta peer-peer system, April 2002. 
http://www.jxta.org/. 

11.	Kazaa Media Desktop.
http://www.kazaa.com/

12.	Aberer, K., and Despotovic, Z.
Managing trust in a peer-to-peer information system.
In ACM Conference on Information and Knowledge Management, 2001.

13.	Reiter, M., and Rubin, A.
Crowds: anonymity for Web transactions.
In Proc. ACM Transactions on Information and System Security, 1998.