Armenian Knowledge Base  

Go Back   Armenian Knowledge Base > Technical sections > Webmaster Zone > Showcase
Register

Reply
 
LinkBack Thread Tools
Old 31.05.2002, 14:45   #1
Младенец
 
Join Date: 04 2002
Location: Russia
Posts: 56
Downloads: 0
Uploads: 0
Reputation: 0 | 0
Post Once again about robots.txt

At first Sorry za offtopic ... chestno govorya ne znal v kakom razdele napisat' ... -(

Rech o robots.txt file-e...

Delo v tom, chto chast' site-a naxoditsya za predelami servera ...(na free servere). krome etogo, na site-ax ne malo text-a s ispolzovaniyem java. Naskol'ko ya znayu, roboti ne lyubyat java and tem bolee razniye servera.

Kak mogno napisat' v robots.txt, chtobi robot prosledoval eshe i na drugoy server...
Nadpis':

Allow: http://www...../

naskol'ko ya mogu sudit' ne rabotaet... da i

allow: /lyuboy folder/ - (v predelax servera)toje

robot validatori pokrayney mere dayut error

Thanx
__________________
Armenian Medical Network
http://www.dental.am
Reply With Quote
Old 01.06.2002, 03:30   #2
Профессор
 
Join Date: 01 2002
Location: New York, USA
Posts: 2,938
Downloads: 0
Uploads: 0
Reputation: 0 | 0
Post

Quote:
Originally posted by Davi Menasaka:
Kak mogno napisat' v robots.txt, chtobi robot prosledoval eshe i na drugoy server...
Naskol'ko mne izvestno robots.txt mozhet otnositsya tol'ko k konkretnomu hostu.

Ya vstrechal v metategax ROBOTS parametr "follow" chto oznachayet vsego lish rekommendaciyu dlya robota posledovat' po linkam.

Na free servere, gde net dostupa k file-u robots.txt, mozhno ispol'zovat' eti metategi.

No eshe raz, eto vsego lish rekomendacii, t.e. crawler mozhet im i ne posledovat'.

Hope this was somehow useful
__________________
Karen Vrtanesyan, աջակցող

ArmenianHouse.org - Armenian Library and Forum.
Literary Cafe - Young Armenian writers and poets
Reply With Quote
Old 01.06.2002, 04:12   #3
Младенец
 
Join Date: 04 2002
Location: Russia
Posts: 56
Downloads: 0
Uploads: 0
Reputation: 0 | 0
Post

thanks
A pochemu na free serverax netu dotupa k robots.txt ...(may be iz za togo, chto robot budet proveryat' http://www.freeserver.com/robots.txt, a ne http://www.***.freeserver.com/robots.txt) ?? da?
ok. ya ponyal...
no rech shel ne o tom, chtobi stavit' na free servere, an osnovnom hoste.... sudya po tvoim slovam eto ne real'no.
eshe... a kak emu ukazat', chtobi on proshel v konkretnii folder... naskol'ko ya ponyal Allow: /syudi/ ne rabotaet..
Neujeli ya mogu tol'ko zapretim (disallow) v konkretnii folder?
Reply With Quote
Old 01.06.2002, 08:05   #4
Профессор
 
Join Date: 01 2002
Location: New York, USA
Posts: 2,938
Downloads: 0
Uploads: 0
Reputation: 0 | 0
Post

Naskol'ko znayu mozhesh' sdelat' tol'ko zapretiv neugodnye (disallow)

Chestno govorya seychas konkretno vsye nyuansy ne pomnyu, no mozhesh' prochitat' tut: http://www.robotstxt.org/
Reply With Quote
Old 01.06.2002, 08:06   #5
Профессор
 
Join Date: 01 2002
Location: New York, USA
Posts: 2,938
Downloads: 0
Uploads: 0
Reputation: 0 | 0
Post

V dogonku:

U tebya problema, chto crawler ne xochet indexirovat' kakoj-to folder?

Mozhesh' popodrobnee opisat' situaciyu?
Reply With Quote
Old 01.06.2002, 10:12   #6
Младенец
 
Join Date: 04 2002
Location: Russia
Posts: 56
Downloads: 0
Uploads: 0
Reputation: 0 | 0
Post

Situation takaya..
Limit arminco davno uje perevipolnili ... prishlos' iskat' hosting, pod ruki popalsya free hosting ot by.ru (napodobiye arminco & bez bannerov, bez limita). seychas gde to 30mb na free hostinge, a na arminco site v osnovnom (nu pokrayney mere menu)sdelan *.js and mnogiye Linki na jave. v itoge crawler indexiruyet max 1/3 or 1/4 chast' site-a.
esli s pomochyu robots.txt ya ne mogu perenapravit' ego na free hosting, tak xotya bi mne xochetsya zastavit' ego proindexirovat' to, chto na arminco.
poetomu mne v golovu prishlo sdelat' site. gde prosto napisat' linki(xotya bi togo, chto na arminco) prosto html-om, and "poprosit'" crawler "allow" to that folder (and site in this folder). Ili prosto (esli est' vozmognost') allow, v te folders, na kotoriye linki sdelanni na jave.
vot takaya xrenoten' (sorry)
Reply With Quote
Old 01.06.2002, 11:49   #7
Профессор
 
Join Date: 01 2002
Location: New York, USA
Posts: 2,938
Downloads: 0
Uploads: 0
Reputation: 0 | 0
Post

1. Один из вариантов перенаправления: поставь несколько НАСТОЯЩИХ (no JS) линков на html страницу с by.ru, на которой будут все линки на другие страницы (это почти то же, что твое: "mne v golovu prishlo sdelat' site. gde prosto napisat' linki").
Только я одного не понял, даже если crawler перейдет на страницы by.ru , то при поиске будет показано только содержание отдельных страниц, то есть навигационная структура будет отсуствовать (если я правильно понял структуру).

2. Делать отдельный перенаправляющий сайт, наверное не стоит, некоторые search engine-ы питают аллергию к doorway сайтам, и могут посчитать это спамом (и соответственно забанить).

Hope this helped.
Reply With Quote
Old 01.06.2002, 22:21   #8
Младенец
 
Join Date: 04 2002
Location: Russia
Posts: 56
Downloads: 0
Uploads: 0
Reputation: 0 | 0
Post

Da, v prinsipe na russian version na pervoy stranise (menu) ya uje sdelal tak. tol'ko vot voznikayet drugaya problema. Site novostnoy, i mne proshe dobolyat'(or sdelat') news(v srednem 13-20) v odnom file (*.js). etot file (js) figuriruet v mnogix ostal'nix stranisax, tak chto estesvenno sdelat' update vsex ostal'nix stranis (html-om) eto i neral'no, and gemoroyno (kakoye interesnoye slovo ).

tak, chto v lyubom sluchaye esli cherez menu (html link) crawler proydyet, to na news(js-om) - takoy variant ne proydyet... - ...
a mne ne menshe nujno, chtobi on proindexiroval news ...(
ladno, poprubuyem v intere nayti drugiye sposobi resheniya etoy problemi.
V lyubom sluchaye bol'shoye tebe spasibo, chto "Uvajil" moy topic
Thanks eshe raz
Reply With Quote
Old 01.06.2002, 23:58   #9
Kooper
 
Kooper_26's Avatar
 
Join Date: 05 2002
Location: Hay.am Portal
Age: 41
Posts: 350
Downloads: 0
Uploads: 0
Reputation: 0 | 0
Arrow

Kogda ya indeksiroval tvoj sajt, naskol'ko ya pomnyu ty robots.txt ne ispol'zoval. A kogda moj crawler zaprashival omlinkax vne hosta, ya obychno razreshal emu ix indeksirovat', esli videl chto oni sootvetstvuyut specifike nashej poiskovoj sistemy. Java cripty crawler vosprinimal normal'no, esli oni staticheskie (ne izmenyayut kontent stranicy posle togo kak ona zagruzhenna). Kstati xochu cherez paru nedel' proizvedu polnuyu pereindeksaciyu, ochen' li mnogo izmennenij na saite???
__________________
---
RGRDS
http://www.hay.am Web Administrator
Reply With Quote
Sponsored Links
Reply

Thread Tools


На правах рекламы:
реклама

All times are GMT. The time now is 14:32.


Powered by vBulletin® Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.