[Frugalware-devel] questions about syncpkgd2

[Gabriel C]

VMiklos wrote:
> Hello,
> 
> Na Mon, Jul 30, 2007 at 10:54:06PM +0200, Gabriel C <crazy at frugalware.org> pisal(a):
>>> 1) still allow this feature (uploading packages manually and
>>> automatically, too) or
>>>
>>> + no big change, does not violate freedom
>>> - when will we build a pkg? if no upload with in ie 2 hours? that will
>>>   slow down syncd again
>> Really what do you mean by *slow down* ?! 
> 
> currently if you're lucky, your pkg is built in 10 mins, if not then it
> may take a half of a day. this is not a big issue if we can upload
> packages manually (imagine a package that somehow does something really
> mad - ie rm -rf important files or whatever - then waiting a half of a
> day for a fix is quite annoying)

Well is why I'm saing *only give power to build tools is not good* sometimes the _humans_ have to quick fix things some
script can't do.

( Anyway I still not see the slow down issue but I may be to tired )

> 
>>   - if syncd is acting up ( deleting packages or whatever else ) we are screwed
> 
> this is already an issue, if for some reason syncd decides to rm *.fpm,
> then it'll be mirrored out and we'll loose our fpms. in fact of course
> this is already handled very seriously and it never happened

Yes but now we can at least stop it to find the bugs and _build_ and _upload_ the missing packages
which is no longer possible if you disallow that for devels
 
> 
>>   - in 'long-term' no one will even *try* to build any package local resulting in :
>> 	-- s/1.2/1.2.1/ && push out
>> 		which means again :
>> 			-- broken and not tested packages _at_ all
> 
> this does not mean you should not build & test packages, it just means
> if you somehow did some trick (you build not in a clean chroot, etc),
> then it won't affect the uses as the packages are built automatically. i
> think this also improves security: if one does something bad by
> accident, then it _will_ have some log in the scm, ie you can't do
> something like:
> 
> 1) modify the FB and cause some silly bug
> 2) build the fpm
> 3) fix the silly bug
> 4) push
> 
> or:
> 
> 1) you detect a bug
> 2) rebuild without relbump
> 3) push
> 
> -> other arches won't be fixed
> 
> to sum up: i think building all packages automatically improves the
> quality of packages and also much more secure. and here i'm not talking
> about malicious devs because i still think the team is small enough that
> i trust devs, but i'm talking about accidents. during the past few
> months i can remember at least 2 times i uploaded buggy fpms, which
> included wrong README.Frugalwares, because i thought i know what i'm
> doing, then forgot to clean the chroot after a failed build, built an
> other pkg and voila: the new pkg contained an unrelated readme.fw
> 

Jepp I know but we are humans :P and with time no one will build why not needed but I see your point.

>>> so, what do you think about this? i think in long-term it would be good
>>> to choose 2), that would ensure a higher package quality
>>>
>> Only 2 is not a good idea , giving only some build tools ( scripts etc ) the power to upload fpm's / packages
>> is wrong .. ( of course IMO ) 
>>
>> And I don't agree at all about the 'higher package quality' packages we already check for 'chroot builds'
>> on upload and that is good as is.
> 
> we do not check about clean chroots. and if we deny building in a dirty
> chroot, we make debugging impossible. just the pkgs hitting current
> should be built in a clean chroot

Syncd can't be trusted to always have a clean chroot ( at least not the one we have now )
I remember the time I run it on my box and the chroot broke some times.


> 
>>> in short-term, maybe it would be good to choose 1), but then we need
>>> this ugly "build the pkg if no fpm uploaded in x time" hack, and we
>>> should decide such an interval
>> Why ugly ? and why we need run the daemons all xx minutes or all some hours ?
>>
>> Run 'twice' a day and is fine for stable and current 
> 
> basically yes, but if syncd build all pkgs then waiting half a day for
> an important fix may be too much
> 

I mean twice a day if we go only for 1 , like is now maybe with some changes.

> to sum up: you say would it be good to stay at 1) and build packages
> automatically if we waited at least 12 hours after the push (in case the
> pkg is uploaded manually till then)?
> 
> feel free to correct me, i just want to sum up our ideas so that
> hopefully we'll have a constructive discussion, not a flame :)

I don't want to flame at all , why ? 

I'm saying only solution 2) is not good. ( at least not good till we are _sure_ the daemons are rock solid )

Auto tools are fine but they easy can break down. So :

-- what we do if syncd dies ?!?
-- what we do if dies while bugs / weird bugs and we need debug it first before start it ?
-- <some other things I can't think of while really tired but I hope you got what I mean>

We have to wait so long till we find the bug(s) ( minutes to days )
In this time no build nor syncing the arch is possible ( worse )

So basically a mix of 1 and 2 would be fine first. Or 2 with 1 as fall back ( eg some flag to still upload for devels if needed ).

/me is really of to bed now ;)

> 
> thanks,
> - VMiklos
> 

crazy